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Abstract 

Many  ’’real  world”  decisions  are  made  by  groups.  It  is  rare  that  the  responsi¬ 
bility  for  a  very  important  decision  is  given  to  a  single  decision  maker  with  complete 
authority.  Group  decision  making  adds  both  advantages  to  the  process  as  well  as 
disadvantages.  This  research  examines  the  question:  ’’Are  decisions  made  by  groups 
really  that  much  different  from  the  decisions  made  by  individuals  in  the  group?” 

A  specific  case  study  involving  the  selection  of  the  best  primary  training  aircraft 
type  for  military  pilot  training  is  used  to  examine  this  question.  Fifteen  military  pilots 
with  various  backgrounds  and  experience  levels  participate  as  decision  makers  in  the 
study.  The  decision  analysis  method  of  Value  Focused  Thinking  is  used  to  facilitate 
both  individual  and  group  decision  making  sessions.  Value  hierarchies  are  created  for 
all  sessions,  and  a  set  of  alternatives  is  generated  and  scored.  Kendall’s  Coefficient  of 
Concordance  is  used  to  determine  the  level  of  agreement  between  decisions  made  by 
the  group  and  individual  decision  makers. 
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ANALYZING  INDIVIDUAL  DECISION  MAKING  VERSUS 
GROUP  DECISION  MAKING  FOR  ALTERNATIVE 

SELECTION 

I.  Introduction 

1 . 1  Background 

Many  decisions  in  the  real  world  are  made  by  groups  rather  than  individuals, 
such  as  families,  committees  or  legislatures.  (Legislators  make  the  laws,  some  business 
decisions  are  made  collectively,  monetary  policy  decisions  are  made  by  committee 
rather  than  by  a  single  individual.)  Households  and  firms  are  typically  not  individuals, 
but  groups  of  people  with  a  joint  interest  in  economic  decisions.  Similarly,  political  or 
military  decisions,  as  well  as,  decisions  on  monetary  policy  are  often  taken  by  groups 
rather  than  individuals.  On  the  other  hand,  there  are  also  decisions  which  are  almost 
always  made  by  individuals:  A  consumer  trying  to  make  a  purchase  decision  with 
a  limited  budget;  a  realtor  deciding  to  maximize  his/her  profit;  a  banker  trying  to 
decide  an  optimal  interest  rate  in  the  free  market. 

In  the  held  of  psychology,  there  is  significant  research  for  comparing  individual 
and  group  behavior  and  decision  making  .  The  results  found  in  psychological  research 
can  be  divided  into  two  main  effects.  First,  groups  are  often  assumed  to  make  better 
decisions  than  individuals.  This  is  based  on  that  simple  idea,  which  is  two  (or  more) 
people  know  more  than  one.  Second,  groups  often  make  different  decisions  than 
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individuals.  From  an  individual’s  point  of  view,  it  can  be  simply  assumed  that  an 
individual  acts  according  to  his/her  own  preferences.  On  a  group  level,  the  choices 
depend  not  only  on  how  the  individual  preferences  of  the  group  members  are  combined 
into  a  group  decision,  but  also  how  they  depend  on  individual  preferences. 

Do  group  decisions  really  differ  a  lot  from  the  decisions  of  the  individuals  who 
form  the  group?  Most  of  the  opinions  agree  that  groups  make  decisions  more  slowly 
than  individuals.  Then  why  are  so  many  important  decisions  made  by  groups?  It  is 
because  of  some  belief  in  common  sense:  It  may  be  advantageous  to  bring  more  than 
one  person  into  consider  a  question  a  complicated  situation  where  no  one  knows  the 
’’true”  way  to  make  a  decision,  or  where  data  may  be  hard  to  process  or  interpret, 
and  where  individual  values  may  influence  decisions. 

1.2  Problem  Statement 

Group  decision  making  is  an  important  and  controversial  part  of  life.  Both 
in  business  and  day-to-day  life,  by  definition,  group  decisions  are  being  made  by  a 
number  of  individuals  coming  together.  A  portion  of  the  research  asserts  that  there 
are  advantages  in  having  many  minds  focused  on  the  same  problem,  and  that  provides 
this  grouping  with  increased  morale.  Another  portion  complains  about  the  slowness 
of  the  group  decision  process,  the  conservatism  and  excessive  caution  of  groups,  and 
the  difficulties  of  determining  who  is  ’’really”  responsible  for  a  decision  made  by  a 
group. 
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The  widely  held  belief  that  groups  reach  more  rational  or  better  decisions  than 
individuals  is  far  from  being  confirmed  by  psychological  literature.  Davis  stated  that 
”in  the  idealized  form  of  the  group  superiority  argument,  groups  are,  considered  to 
balance  biases,  catch  errors  and  stimulate  thoughtful  work”  [8].  The  conventional 
wisdom  of  group  superiority  has  been  challenged  by  numerous  experiments,  leading 
to  the  conclusion  that  ’’group  discussion  can  decrease,  amplify,  or  simply  reproduce 
the  judgmental  bias  of  individuals”  as  indicated  by  Holloman  and  Hendrick  [5]. 

Although  many  studies  have  been  performed  in  the  area  of  individual  decision 
making,  the  studies  do  not  thoroughly  explain  the  relative  relation  between  groups  and 
individuals.  The  general  social  psychological  literature  does  not  offer  an  unambiguous 
line  of  reasoning  which  would  help  to  predict  the  relationship  of  individuals  and 
groups. 

Group  decision  making  involves  different  characteristics  in  which  bad  decisions 
are  likely  to  be  made.  Some  decision  makers  may  avoid  expressing  their  true  opinion 
on  the  matter  at  hand  for  fear  of  stepping  outside  the  comfort  zone  of  the  group. 
Some  members  of  the  group  may  feel  reluctant  to  share  their  opinions  and  fear  a  loss 
of  respect  or  job  title  for  speaking  the  wrong  words.  Johnson  and  Libecap  state  that 
’’group  pressures,  biases,  and  other  behaviors  reduce  the  quality  of  the  decision”  [31]. 

In  the  stressful  and  pressure-filled  atmosphere  of  group  decision  making,  if  many 
group  members  are  likely  to  hesitate  in  expressing  their  real  values,  ideas  and  prefer¬ 
ences,  is  it  absolutely  necessary  and  efficient  to  gather  a  number  of  people  together? 
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If  the  individuals  can  already  make  the  same  decisions  with  the  same  or  maybe  even 
higher  efficiency,  why  would  the  valuable  time  of  many  participants  be  consumed  in 
a  group  decision  making  environment? 

The  case  study  will  be  proceeded  with  a  simple  question  rather  than  a  complex 
hypothesis.  The  problem  to  be  explored  is:  ’’Are  decisions  made  by  groups  really 
that  different  from  the  decisions  made  by  individuals?”  A  slightly  different  focus  can 
also  be  employed.  It  asks  the  question:  ”Do  individuals  really  conduct  themselves 
differently  when  they  make  decisions  a  member  of  a  group  or  when  they  reach  decisions 
by  themselves?” 

1.3  Research  Focus  &  Methodology 

A  specific  case  study  regarding  what  is  important  to  a  pilot  in  a  primary  train¬ 
ing  aircraft  was  used  to  examine  whether  the  decisions  made  by  groups  are  really  that 
much  different  from  the  decisions  made  by  individuals.  This  case  study  was  performed 
by  questioning  fifteen  decision  makers  (DMs),  who  also  contributed  as  subject  matter 
experts  (SME)  from  a  wide  variety  of  sources  (Air  Force,  Navy,  Marine  Corps,  In¬ 
ternational  Pilots),  on  what  is  valued  in  a  primary  training  aircraft.  Specifically,  the 
pilots  were  selected  as  DMs  since  they  are  the  ’’experts”  and  know  best  in  deciding 
what  is  important  in  a  primary  training  aircraft  in  which  they  either  have  flown  or 
are  still  flying.  The  group  of  DMs  consisted  of  US  Air  Force  pilots  (  2  Lt.Coh,  5 
Maj.),  US  Navy  pilots  (  3  LCDR)  ,  US  Marine  Corps  pilot  (  1  Maj.)  and  Turkish 
Air  Force  pilots  (4  First  Lt.s).  The  experience  levels,  ranks,  ages,  and  proficiency  in 
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aircraft  of  the  DMs  varied  tremendously.  The  military  forces  of  the  DMs  were  also 
scattered.  This  wide  variety  group  of  DMs  provided  the  study  with  a  decision  at 
the  end.  After  creating  value-focused  thinking  (VFT)  hierarchies  for  each  DM  and 
the  group,  a  list  of  alternatives  has  been  scored  based  on  each  hierarchy.  This  score 
provided  the  means  to  examine  and  explore  the  differences,  if  any,  among  the  fifteen 
DMs  and  group.  No  difference  would  verify  that  individual  decision  making  would  be 
at  least  the  same  efficient  or  more  efficient  than  bringing  the  group  together  to  make 
the  decision. 

Whether  there  is  an  association  among  the  decision  makers  were  determined  in 
the  final  result  section.  The  DMs  have  been  grouped  under  different  sample  forms 
,  such  as  all  DMS  &  group,  USAF  DMs  &  group,  USN  &  USMC  DMs  &  group, 
TUAF  DMs  &  group,  instructor  DMs  &  group,  non- instructor  DMs  &  group  and  null 
hypothesis:  ”m  sets  of  rankings  are  independent”  have  been  tested  to  see  if  there  is 
an  agreement  among  the  DMs  for  the  statistical  analysis.  The  result  of  the  study 
helps  allay  the  stress  of  group  meetings,  the  time  a  decision  takes  to  be  met,  and  the 
perceived  influence  DM’s  have  on  one  another. 

The  goal  of  the  research  is  to  determine  if  there  is  really  a  difference  between 
an  individual  and  group  decision  on  the  same  topic. 

Value  Focused  Thinking  (VFT)  technique  has  been  applied  as  the  methodology. 
VFT  focuses  on  the  values  of  the  decision  maker  rather  than  the  alternatives  that  are 
already  available. 
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All  value  hierarchies  were  created  and  VFT  process  has  been  implemented  by 
using  Lt.  Col.  Jeffrey  D.  Weir’s  ’’Hierarchy  Builder”  software  program. 

The  remainder  of  this  document  is  consisting  of  four  chapters  and  4  appendices. 
Chapter-2  reviews  literature  covering  group  decision  making,  advantages  &  disad¬ 
vantages,  individual  versus  group  decision  making,  case  study,  decision  analysis  and 
VFT.  Chapter-3  provides  detailed  information  on  the  methodology,  step-by-step,  as 
the  research  is  conducted  and  develops  value  models.  Chapter-4  runs  the  value  mod¬ 
els  to  evaluate  and  displays  the  results  of  the  analysis  done  with  the  process  outlined 
in  Chapter-3.  Chapter-5  presents  the  results  of  the  study  and  the  limitations  of  the 
work,  provides  discussion  on  recommendations  based  on  the  results,  as  well  as  future 
research  that  can  be  done  to  further  explore  the  issue. 
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II.  Literature  Review 


2.1  Overview 

This  chapter  discusses  group  decision  making,  individual  versus  group  decision 
making,  the  case  study  question,  decision  analysis,  and  value  focused  thinking.  Deci¬ 
sion  analysis  is  discussed,  and  an  in-depth  discussion  is  provided  on  the  VFT  approach 
to  decision  making. 

2.2  Group  Decision  Making 

Many  people  come  together  everyday  to  make  decisions  concerning  some  topic 
of  interest  for  the  group  or  a  group  of  people  that  share  common  characteristics  or 
interests.  Group  decision  making  is  a  process  that  has  a  very  wide  range  of  ideas 
and  events.  It  is  a  type  of  participation  process  in  which  multiple  individuals  act  as 
a  group,  analyze  problems  or  situations,  consider  and  evaluate  alternative  courses  of 
action,  and  select  from  among  the  alternatives  a  solution  or  solutions.  The  nature 
of  the  decision  can  be  a  high  ranking  political  issue,  or  it  can  also  be  a  public  jury 
deciding  whether  someone  is  guilty  or  not.  But  it  can  equally  be  unofficial  with  some 
decision  on  the  room  setting.  Deciding  whether  someone  is  guilty  or  not  and  the  deci¬ 
sion  for  the  setting  of  a  room  are  of  a  totally  different  nature.  The  number  of  people 
involved  in  group  decision  making  varies  greatly.  The  individuals  in  a  group  may  be 
demographically  similar  or  quite  diverse.  The  character  and  composition  of  groups, 
their  size,  and  demographic  structure  all  affect  their  performance  to  some  degree.  In 
addition  to  these,  the  external  pressure  faced  by  groups  such  as  time  demands  and 
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conflicting  goals  may  also  impact  the  progress  and  effectiveness  of  decision  making 


groups. 

Groups  are  nothing  more  than  a  gathering  of  people  who  are  working  toward 
the  same  goals,  in  the  same  place,  at  a  given  time.  However,  when  people  meet  in  this 
way,  surprising  changes  in  the  behavior  of  individuals  emerge.  As  explained  by  Latane 
in  his  Social  Impact  Theory,  the  total  impact  of  other  people  in  an  individual  is  related 
to  the  the  number,  strength,  and  directness  of  other  observers.  When  observers  have 
a  higher  level  of  power  (determined  by  status,  age,  relationship),  or  are  greater  in 
number,  there  will  be  a  greater  effect  on  the  individual’s  behavior  [27]. 

The  domain  of  decision  making  is  vast.  The  structure  of  the  group  has  diverse 
effects  on  group  decision  making  as  well.  It  is  still  open  whether  group  decision  making 
produces  worthwhile  results  for  a  decision.  This  can  depend  on  many  different  factors 
such  as  each  person’s  personality,  the  state-of-mind  at  the  moment  of  the  decision,  the 
structure  of  the  group,  and  the  ideas  behind  groupthink.  Brahm  and  Kleiner  express 
that  ’’often,  members  do  not  feel  comfortable  expressing  themselves  verbally,  even  in 
groups  structured  to  be  smaller”  [3].  This  can  cause  some  members  to  sit  in  a  group 
meeting  and  never  voice  their  own  opinions  on  the  decision  making  process,  therefore, 
decreasing  the  need  for  them  to  be  present.  They  also  stated  that  ’’the  decision  makers 
of  the  group  must  be  assertive  and  participate  enough  to  contribute  to  the  group”  [3] . 
The  presence  of  a  supervisor  can  have  very  negative  effects  on  the  process  because 
they  have  the  final  say  on  whether  the  proposed  decision  is  acceptable.  They  either 


accept  the  idea  or  reject  it,  and  thereby  that  result  in  demoralization  of  the  group 
members.  An  unhappy  group  ends  up  to  be  an  unproductive  group. 

Individual  influences  are  reflected  in  group  decisions.  The  evidence  on  whether 
groups  or  individuals  make  better  decisions  is  mixed.  Kerr,  Kramer,  MacCoun  empha¬ 
sized  that  ’’the  details  of  the  group- judgment  process  are  an  important  determinant 
of  the  quality  of  the  group  versus  individual  decisions”  [30].  Wallach,  Kogan,  and 
Bern  have  found  that  ’’group  decisions  can  lead  to  excessive  risk  taking  (so-called 
risky  shift)”.  Bornstein  and  Yaniv  studied  individual  versus  team  choices  in  ultima¬ 
tum  games  and  found  that  ’’teams  are  more  game-theoretically  rational  players  than 
are  individuals”  [11]  .  In  contrast,  Cox  and  Hayne  found  that  ’’groups  tend  to  de¬ 
viate  further  from  balanced  strategies  than  do  individuals”  [16].  Kocher  and  Sutter 
compared  decisions  by  individuals  and  groups  in  beauty  contest  games  [29].  They 
found  no  difference  in  the  depth  of  reasoning  between  individuals  and  groups,  but 
found  that  groups  learn  faster  than  individuals.  Michaelsen,  Watson,  and  Black  de¬ 
termined  that  there  have  been  no  studies  on  individual  versus  group  decision  making 
that  provided  any  significant  outcome,  positive  or  negative,  for  either  individual  or 
group  performance  [26].  It  is  a  big  question  if  the  decisions  made  by  groups  differ 
from  the  decisions  of  the  individuals. 

A  strong  personal  identity  in  a  group  could  lead  to  different  results.  In  some 
cases  personal  identity  is  lost  and  replaced  by  an  identification  with  the  goals  and  ac¬ 
tions  of  the  group  which  is  called  the  ’’deindividuation”  described  by  Gustave  LeBon 
in  his  Crowd  Theory  (1896)  [28].  He  developed  a  theory  that  ”in  the  right  situations, 
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the  emotions  of  one  person  spread  through  the  group  like  a  cold  through  a  building” . 
According  to  the  theory,  during  the  time  shared  in  a  group  environment,  control 
mechanisms  such  as  values,  principles,  and  learned  social  rules  are  broken  down  and 
forgotten.  Forsyth  defines  ’’the  complete  feeling  of  anonymity  and  being  less  account¬ 
able”  as  the  key  to  people’s  actions  [10].  Many  other  theorists  (Zimbardo,  1970; 
Diener,  Fraser,  Bearnen,  and  Kelem,  1976)  have  done  experiments  showing  that  when 
the  sense  of  responsibility  for  onr  actions  is  lost,  the  participants  will  behave  in  more 
extreme  ways  then  when  each  member  feels  responsibility  for  the  decision.  [40]  [9]. 

Agreeing  with  the  beliefs  of  a  group  may  affect  personal  decisions.  Groups 
make  decisions  that  are  often  riskier  than  what  would  be  expected  given  the  views 
held  by  individual  members  before  the  decision.  People  have  a  tendency  to  compare 
themselves  socially  to  the  opinions  of  others  in  the  group.  If  the  group  tends  to  be 
leaning  to  a  more  radical  idea,  many  others  will  also  shift  their  views  in  that  direction. 

Group  decision  making  is  found  in  many  different  areas  of  the  workforce.  People 
take  different  sides  on  the  issue  of  the  effectiveness  of  group  decision  making.  Watson, 
Sharp,  and  Michaelsen  have  debated  that  group  decision  making  is  effective  and  an¬ 
nounced  that  ’’decisions  made  will  reflect  those  higher  qualities  than  the  group’s  most 
knowledgeable  member”  [38].  This  coincides  with  the  notion  that  different  values  or 
alternatives  may  come  about  in  a  group. 
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2.3  Advantages  and  Disadvantages  of  Group  Decision  Making 

The  effectiveness  of  decision  making  groups  can  be  affected  by  a  variety  of  fac¬ 
tors.  Therefore,  it  is  not  possible  to  suggest  that  group  decision  making  is  always 
better  or  group  decision  making  is  always  worse  than  individual  decision  making.  For 
example,  due  to  the  increased  demographic  diversity  in  the  workforce,  a  considerable 
amount  of  research  has  focused  on  diversity’s  impact  on  the  effectiveness  of  group 
performance.  In  general,  demographic  diversity  can  sometimes  have  positive  or  nega¬ 
tive  effects,  depending  on  the  specific  situation.  Some  research  indicates  that  diverse 
groups  tend  to  generate  a  wider  variety  and  higher  quality  of  decision  alternatives 
than  demographically  homogeneous  groups. 

In  the  ideal,  group  decision  making  takes  advantage  of  the  varied  strengths  and 
expert  opinions  of  its  members.  It  is  possible  that  the  group  can  generate  a  greater 
number  of  alternatives  that  are  of  higher  quality  than  the  individual.  If  a  greater 
number  of  higher  quality  alternatives  are  generated,  then  it  is  likely  that  the  group 
will  eventually  reach  a  better  and  greater  problem  solution  than  the  individual.  Since 
it  is  possible  that  many  of  the  group  members  are  affected  by  the  decision  at  the  end, 
they  actually  might  have  input  into  the  decision  which  results  in  a  group  decision  of 
greater  combined  understanding  of  the  eventual  course  of  action  chosen. 

There  are  also  many  potential  disadvantages  to  group  decision  making.  Groups 
are  generally  slower  to  arrive  at  decisions  than  individuals,  so  sometimes  it  is  diffi¬ 
cult  to  put  them  to  practical  use  in  situations  where  decisions  must  be  made  very 
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quickly.  One  of  the  most  common  problems  is  ”  groupthink” .  Irving  Janis  defined  the 
phenomenon  as  ’’the  decline  of  mental  efficiency,  reality  testing,  and  moral  judgment 
resulting  from  group  pressure”  in  his  book  ’’Victims  of  Groupthink”  [18]. 

Groupthink  is  a  characteristic  of  group  decision  making  in  which  bad  decisions 
are  likely  to  be  made.  Groupthink  can  be  thought  of  as  a  mode  of  thinking  that 
people  engage  in  when  they  are  deeply  involved  in  a  well-united  group.  Groupthink 
occurs  when  individuals  in  a  group  feel  pressure  to  comply  with  what  seems  to  be  the 
dominant  view  in  the  group.  Opposing  views  of  the  majority  opinion  are  suppressed, 
and  alternative  courses  of  action  are  not  fully  explored.  Some  decision  makers  avoid 
raising  their  true  opinion  on  the  matter  at  hand  for  fear  of  stepping  outside  the 
comfort  zone  of  the  group.  Members  of  the  group  may  feel  inferior  to  others  and 
fear  a  loss  of  respect  for  speaking  the  wrong  words.  Johnson  indicates  that  those 
group  pressures,  biases,  and  other  behaviors  reduce  the  quality  of  the  decision.  He 
also  states  that  ’’the  decision  context  and  available  alternatives  will  determine  the 
effect  of  groupthink  on  a  situation”  [21].  Although  groupthink  in  different  situations 
can  be  difficult  to  detect  or  measure,  it  offers  insight  into  understanding  how  various 
factors  and  conditions  combine  to  affect  decision  outcomes.  To  prevent  groupthink 
and  enhance  the  effectiveness  of  group  decision  making,  the  leader  of  the  group  may 
encourage  the  group  members  to  express  objections,  or  he  or  she  may  also  remain 
unbiased  to  ideas  as  they  are  presented. 

Some  research  suggests  that  certain  characteristics  of  groups  contribute  to  group¬ 
think.  In  the  first  place,  if  the  group  does  not  have  an  agreed  upon  process  for  devcl- 
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oping  and  evaluating  alternatives,  it  is  possible  that  an  incomplete  set  of  alternatives 
will  be  considered  and  that  different  courses  of  action  will  not  be  fully  explored.  Many 
of  the  formal  decision  making  processes,  for  instance  ”  nominal  group  technique”  and 
’’brain-storming”,  are  designed  to  reduce  the  potential  for  groupthink  by  ensuring 
that  group  members  offer  and  consider  a  large  number  of  decision  alternatives.  Sec¬ 
ondly,  if  a  powerful  leader  dominates  the  group,  other  group  members  may  quickly 
comply  with  the  dominant  view.  Additionally,  if  the  group  is  under  stress  and/or  time 
pressure,  groupthink  may  occur.  Finally,  studies  suggest  that  well-adjusted  groups 
are  more  susceptible  to  groupthink. 

Another  potential  disadvantage  of  group  decision  making  is  group  polarization. 
The  group  my  tend  to  focus  on  more  extreme  solutions  to  a  problem.  An  example 
of  polarization  is  the  risky  shift  phenomenon.  When  the  group  decision  is  a  riskier 
one  than  any  of  the  group  members  would  have  made  individually,  it  results  in  risky 
shift.  Members  of  the  groups  sometimes  may  not  feel  as  much  responsibility  for  the 
actions  of  the  group  as  they  would,  if  they  were  making  the  decision  alone. 

Decision  making  in  groups  is  a  fact  of  life  in  organizations  for  many  individuals. 
Because  so  many  individuals  spend  at  least  some  of  their  work  time  in  decision  making 
groups,  hundreds  of  research  studies  are  carried  out  on  such  groups  each  year.  There 
is  still  much  to  learn  about  the  development  and  performance  of  groups.  However, 
it  is  more  likely  that  research  studies  will  continue  to  focus  on  identifying  course  of 
actions  that  will  make  group  decision  making  more  efficient  and  effective.  It  is  also 
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likely  that  the  effect  of  internal  characteristics  of  groups  (e.g  demographic,  age,  status, 
rank  etc.)  will  be  researched  more  in  depth. 

2.4  Individual  Versus  Group  Decision  Making 

Individual  decision  making  occurs  happens  a  lot  on  a  daily  basis  (financial  in¬ 
vestments,  residential  location,  insurance  etc.).  But  decisions  are  often  taken  by  a 
group  rather  than  by  a  single  individual.  Even  when  these  decisions  are  formally  taken 
by  only  one  of  the  members  of  the  group,  they  may  affect  (and/or  be  affected  by)  the 
way  the  group  shares  other  decisions.  Almost  every  important  decision  involves  risk 
and  individuals  include  risk  into  their  decisions. 

It  can’t  be  simply  assumed  that  groups  make  the  same  decisions  as  individuals. 
There  is  a  significant  amount  of  research  in  psychology  comparing  individual  and 
group  behavior  and  decision  making.  The  psychological  findings  can  be  very  roughly 
divided  into  two  main  effects.  First,  groups  are  often  assumed  to  make  better  decisions 
than  individuals.  This  is  based  on  the  simple  idea  that  two  (or  more)  people  know 
more  than  one.  This  effect  depends  on  the  decision  at  hand;  there  has  to  be  a  provable 
right  solution.  Secondly,  groups  often  make  different  decisions  than  individuals.  On 
an  individual  level,  a  particular  individual  acts  according  to  his/her  preferences  (if 
someone  prefers  X  over  Y,  he/she  will  choose  X).  On  a  group  level,  choices  not  only 
depend  on  how  the  individual  preferences  of  the  group  members  are  combined  into 
a  group  decision,  but  individual  preferences  can  change  because  an  individual  is  a 
member  of  a  particular  group. 
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A  growing  amount  of  literature  has  concentrated  on  the  impact  of  different 
characteristics  of  the  decision  maker,  among  which  the  differences  between  male 
and  female  decision  makers  have  been  most  thoroughly  studied  (e.g.,  Eckel  and 
Grossman, 1998  [4];  Andreoni  and  Vesterlund,2001  [15];  Gneezy,  Niederle  and  Rus- 
tichini,2003  [37]). 

Bornstein  and  Yaniv  have  studied  individual  versus  group  behaviour  in  a  stan¬ 
dard,  ” one-shot  ultimatum  game”,  where  a  fixed  amount  of  money  is  split  between  a 
proposer  and  a  responder  [11]  .  Bornstein  and  Yaniv  compared  two  treatments,  one 
with  individuals  playing  against  individuals  and  one  with  groups  (of  three  subjects 
each)  playing  against  groups.  Their  main  result  was  that  ”  groups  are  more  rational 
players  than  individuals  by  demanding  more  than  individuals  in  the  role  of  proposer 
and  by  accepting  relatively  lower  offers  in  the  role  of  responder”. 

Cox  and  Hayne  have  explored  decision  making  of  groups  and  individuals  in 
common  value  auctions,  characterized  by  ’’risky  outcomes”  [17].  ’’Although  both 
groups  and  individuals  deviate  from  rational  bidding  when  they  have  more  informa¬ 
tion,  groups  are  more  affected  by  the  disadvantage  of  information,  leading  to  the 
conclusion  that  groups  are  less  rational  decision  makers  than  individuals”.  The  stud¬ 
ies  of  Bone,  Hey  and  Suckling  [20]  and  Rockenbach,  Sadrich  and  Mathauschek  [2]  are 
somewhat  related  to  the  paper  by  Cox  and  Hayne  in  that  they  investigated  group 
decision  making  under  risk. 
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Blinder  and  Morgan  studied  group  versus  individual  decision  making  in  an  urn 
problem  and  in  a  monetary  policy  experiment  [1].  Blinder  and  Morgan  were  partic¬ 
ularly  interested  in  whether  groups  are  slower  in  decision  making  than  individuals. 
Measuring  the  speed  of  decision  making  by  the  amount  of  information  needed  before 
reaching  a  decision,  they  found  no  support  for  the  widely  held  belief  that  it  takes 
longer  for  groups  to  reach  a  decision. 

Gillct  explored  the  view  that  ’’groups  are  better  than  individuals  at  solving  an 
inter-temporal  choice  problem” ,  which  he  saw  as  confirmation  for  the  hypothesis  that 
groups  are  smarter  than  individuals,  by  designing  and  running  a  number  of  sepa¬ 
rate  experiments  combining  different  attributes  to  compare  the  differences  between 
decisions  made  by  groups  and  individuals  [12]. 

Finally,  Kocher  and  Sutter  contributed  to  this  growing  literature  availability  by 
addressing  the  influence  of  the  type  of  decision  maker  on  the  rationality  of  decision 
making  [29].  They  focused  on  two  main  research  questions.  First,  ’’are  groups  of  more 
rational  decision  makers  in  the  sense  that  their  decisions  are  closer  or  converge  faster 
to  the  balanced  prediction  of  the  strategic  situation?”  Second,  ”do  groups  outper¬ 
form  individuals  in  terms  of  payoff  when  competing  against  individuals?”  They  used 
an  experimental  beauty-contest  game.  Their  first  research  question  was  related  to  a 
growing  number  of  economic  literature  examining  whether  groups  behave  and  decide 
differently  than  individuals.  An  interesting  general  result  was  the  fact  that  ’’differ¬ 
ences  in  decision  making  between  groups  and  individuals  can  neither  be  explained  by 
simple  collection  of  individual  preferences  or  choices  nor  by  simple  theories  of  group 
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decision  making”.  They  came  up  with  the  conclusion  that  ”it  seems  reasonable  that 
many  important  and  recurrent  decisions  in  societies  are  entrusted  to  groups” .  It  may 
be  beneficial  to  use  groups  as  decision  makers  instead  of  individuals.  However,  they 
also  stated  that  they  still  know  too  little  on  such  important  questions  as,  for  instance, 
which  tasks  should  actually  be  entrusted  to  small  groups  as  decision  makers  or  which 
internal  structure  of  small  groups  contributes  best  to  reaching  optimal  decisions  and 
preventing  adverse  effects  like  groupthink  or  overconfidence  biases. 

2.5  Case  Study  :  Best  Primary  Training  Aircraft 

Pilot  training  aircraft  are  used  to  take  officers  through  flight  training  programs 
to  provide  rated  military  aviators  to  fly  their  respective  services’  fixed-and  rotary¬ 
wing  aircraft.  According  to  Federation  of  American  Scientists’  (FAS)  military  analysis 
[14],  the  flight  training  process  has  been  shaped  and  conducted  through  the  years  as 
summarized  below: 

”The  Air  Force  has  transitioned  from  undergraduate  pilot  training  (UPT)  to 
specialized  undergraduate  pilot  training  (SUPT)  in  order  to  better  prepare  pilots  for 
the  entire  spectrum  of  aircraft  and  flying  missions.  Specialized  undergraduate  pilot 
training  began  at  Reese  Air  Force  Base,  Texas,  in  July  1992,  following  the  arrival  of 
the  T-1A  aircraft.  Undergraduate  pilot  training,  which  universally  trained  all  students 
in  the  T-37  and  T-38  trainer  aircraft,  continued  at  each  base  at  the  same  time  until 
all  required  T-1A  aircraft  arrived  at  that  base.  Transition  to  SUPT  was  completed 
in  early  1997  when  the  last  UPT  class  graduated  at  Columbus  AFB,  Miss. 


17 


Specialized  undergraduate  pilot  training  differs  from  generalized  training  pri¬ 
marily  in  the  advanced  phase.  After  primary  training  in  the  T-37  Tweet,  or  the 
U.S.  Navy’s  T-34  Mentor,  students  select,  by  order  of  merit,  advanced  training  in  the 
bomber- fighter,  airlift-tanker,  helicopter,  or  turboprop  tracks. 

The  preflight  phase  of  SUPT  takes  three  weeks  and  consists  of  academics  and 
physiology  training  to  prepare  students  for  flight.  The  second  phase,  primary  training, 
is  conducted  in  the  twin-engine,  subsonic  T-37  Tweet,  a  rugged  aircraft  equal  in 
maneuverability  to  most  of  the  fighters  of  World  War  II.  Students  learn  aircraft  flight 
characteristics,  emergency  procedures,  takeoff  and  landing  procedures,  aerobatics, 
and  formation  flying.  Students  also  practice  night,  instrument,  and  cross-country 
navigation  flying.  Primary  training  takes  approximately  23  weeks  and  includes  about 
250  hours  of  ground  training,  25  hours  in  the  flight  simulator,  and  90  flying  hours  in 
the  T-37. 

Advanced  training  for  the  bomber-fighter  track  is  accomplished  using  the  T-38 
Talon  and  prepares  pilots  for  transition  to  fighter  and  bomber  aircraft.  The  T-38  is  a 
tandem-seat,  twin-engine,  supersonic  jet.  There  is  increased  emphasis  on  formation, 
navigation,  and  low  level  navigation  flying.  Training  takes  approximately  26  weeks 
and  includes  about  380  hours  of  ground  training,  30  hours  in  the  flight  simulator  and 
120  flying  hours  in  the  T-38. 

The  airlift  tanker  track  uses  the  T-1A  Jayhawk,  the  military  version  of  a  multi¬ 
place  business  jet,  facilitating  the  transition  to  crew  positions  in  airlift  and  tanker 
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aircraft.  Instruction  centers  on  crew  coordination  and  cockpit  management  duties  in 
a  multi-place  crew  aircraft.  Flight  training  includes  visual  and  instrument  transition, 
radar  cell  formation,  and  simulated  refueling  and  airdrop  missions.  Training  takes 
approximately  26  weeks  and  includes  about  185  hours  of  ground  training,  45  hours  in 
the  flight  simulator  and  100  flying  hours  in  the  T-1A. 

The  helicopter  track  trains  in  the  UH-1  Huey  utility  helicopter  for  follow-on 
assignments  in  special  operations,  rescue,  missile  site  support,  and  distinguished  per¬ 
sonnel  support  missions.  The  helicopter  track  transitions  students  from  fixed  wing 
to  rotary-winged  flight.  The  initial  phase  consists  of  basic  helicopter  flying  including 
takeoff  and  landing,  hovering,  and  emergency  procedures.  Advanced  training  consists 
of  instruments,  day  tactics,  and  night  tactics  including  night  vision  goggle  training. 
Training  takes  approximately  24  weeks  and  includes  25  hours  in  a  simulator  and  110 
hours  of  flying  time.” 

The  decision  of  choosing  a  primary  training  aircraft  to  train  new  military  pilots 
to  the  standard  at  which  they  can  go  to  their  next  training  phase  (advanced  training) 
and  then  to  operational  level  units  has  always  been  a  challenging  and  important 
process  for  the  military  forces  throughout  the  world.  The  case  study  focuses  on  this 
problem.  What  are  the  characteristics  of  an  ideal  primary  training  aircraft?  This 
question  has  been  asked  this  way  ’’What  is  important  to  you  in  a  primary  training 
aircraft?”,  to  the  Air  Force,  Navy  and  Marine  Corps  pilots,  who  come  from  different 
backgrounds,  different  ranks,  different  experience  levels,  different  ages  and  different 
perspectives. 
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Primary  training  includes  general  handling,  stalling,  spinning,  solo  flight,  for¬ 
mation  flying,  aerobatics,  and  navigation  at  high  and  low  levels,  and  various  levels 
of  instrument  training.  One  desirable  characteristic  of  a  primary  training  aircraft  for 
relatively  inexperienced  students  is  that  it  must  be  easy  to  fly  safely,  and  to  fly  solo, 
in  clear  air  and  in  clouds,  in  turbulent  and  smooth  air,  in  good  and  bad  weather,  at 
low  level  and  at  high  altitude.  However,  it  should  not  be  too  easy  to  fly  well  enabling 
students  to  be  easily  assessed  for  ability  and  potential.  The  aircraft  must  react  safely 
to  typical  student  mistakes  such  as  flying  slightly  out-of-trim  or  flying  with  less  than 
perfect  accuracy  such  as  on  the  downwind  leg  of  the  traffic  pattern,  in  the  critical 
final  turn  before  landing,  in  night  flying  or  flying  at  low  level.  Also,  when  making 
predictably  large  errors,  such  as  in  aerobatics  students  can  make  incredibly  creative 
mistakes.  In  such  an  aircraft,  natural  aerodynamic  stability  and  ease  of  trimming  are 
requirements  rather  than  having  a  training  aircraft  that  flies  like  a  neutrally  stable 
fighter  and  therefore  difficult  for  student  pilots  to  fly. 

The  priority  in  choosing  the  characteristics  of  any  training  aircraft  is  always 
how  students  will  fly  it,  not  how  instructors  and  senior  officers  will  enjoy  it.  The 
second  part  is  important  for  instructor  motivation,  but  teaching  the  student  is  the 
primary  aim.  Although  this  is  perhaps  obvious,  it  is  often  forgotten  because  the  pilots 
who  assess  training  aircraft  before  purchase  (senior  officers  who  like  playing  with  the 
aircraft  around  the  sky,  which  is  a  great  relief  compared  to  their  desk  work  for  sure) 
will  naturally  be  very  experienced.  It  is  sometimes  difficult  for  an  experienced  pilot  to 
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put  himself  back  instead  of  an  inexperienced  student  unless  he  is  a  current  instructor 
with  a  real  feeling  for  the  failings  of  the  student  pilots. 

The  study  includes  many  terms  relating  to  the  aircraft  systems.  All  those  terms 
are  explained  in  Appendix-B:  Aircraft  Systems  Terminology. 

The  pilots  (decision  makers)  participating  in  this  case  study  have  mentioned 
several  criteria  which  forms  their  values,  such  as  reliability,  performance,  maintain¬ 
ability,  technology,  safety,  flying  quality,  training  quality,  design,  supporting  systems 
and  cost,  which  a  primary  training  aircraft  should  possess  in  order  to  guarantee  im¬ 
provement  in  the  skills  of  the  trainees.  These  values  also  branch  out  to  the  sub-tier 
values.  In  order  to  familiarize  individuals  with  the  terms  and  expressions,  these  cri¬ 
teria  were  explained  in  detail  in  Appendix-C:  The  Decision  Makers’  Criteria. 

2.6  Decision  Analysis 

Clemen  states  that  ’’the  obvious  reason  for  studying  decision  analysis  is  that 
carefully  applying  its  techniques  can  lead  to  better  decisions”  [6].  Each  individual 
decision  has  its  own  defining  frame.  The  decision  analysis  process  helps  the  decision 
maker  pick  the  best  alternatives  within  the  decision  frame.  ”A  decision  is  considered 
difficult  due  to  its  complexity,  uncertainty  in  the  situation,  multiple  objectives,  differ¬ 
ent  perspectives  and  different  conclusions”  [6].  Decision  analysis  provides  structure 
and  guidance  for  systematic  thinking  in  difficult  situations  [6].  Decision  analysis  in¬ 
cludes  many  procedures,  methods,  and  tools.  It  helps  recognizing,  clearly  depicting, 
and  formally  estimating  the  important  aspects  of  a  decision  situation  for  addressing 
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the  recommended  course.  However,  decision  analysis  is  not  designed  to  make  the 
decision  for  the  decision  maker. 

The  Webster  Dictionary  defines  analysis  as  ” separation  of  a  whole  into  its  com¬ 
ponent  parts” .  Decision  analysis  is  the  process  of  separating  a  complex  decision  into 
its  component  parts,  ft  is  a  method  of  helping  decision  makers  make  simple  and  fa¬ 
miliar  choices.  It  is  using  a  mathematical  model  to  conclude  from  these  choices  what 
would  the  decision  maker  would  have  preferred  to  do  in  a  complex,  inflexible  decision. 
Any  real  world  decision  has  many  different  effects  which  are  important  dimensions 
that  should  be  considered  during  the  analysis  and  in  any  assessment  performed  after¬ 
ward. 

In  the  decision  analysis  literature,  there  are  many  methodologies  to  help  one 
make  logical  decisions.  In  most  methodologies,  there  is  a  repeating  topic  of  breaking 
down  or  decomposing  the  problem  to  better  understand  the  situation  and  to  simplify 
the  problem.  Each  process  includes  some  form  of  the  following  steps:  define  the 
problem,  identify  objectives,  develop  alternatives,  evaluate  consequences,  and  evaluate 
tradeoffs.  The  problem  is  the  challenge  that  must  be  solved,  the  objectives  are  the 
desired  goals  for  achievement,  the  alternatives  are  possible  solutions,  the  consequences 
are  the  undesirable  side  effects  of  alternatives,  and  tradeoffs  are  values  that  can  be 
exchanged.  Models  are  developed  in  various  stages  in  the  process,  frequently  involving 
numerical  expressions  to  allow  experts  to  acknowledge  what  is  known  and  not  known. 
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Decisions  are  hard  to  make  and  complex  because  of  uncertainty  about  the  out¬ 
come  of  the  decisions  and  confusion  about  the  value  of  various  outcomes.  The  purpose 
of  analysis  is  not  to  capture  decisions  in  all  their  complexity.  The  goal  of  analysis  is 
to  simplify  the  decision  enough  to  meet  the  decision  maker’s  needs.  An  important 
challenge  then  is  to  determine  how  to  simplify  an  analysis  without  diminishing  its 
usefulness  and  accuracy. 

A  useful  simplification  is  to  ignore  some  uncertainties,  so  the  value  of  an  action 
is  assumed  to  be  more  certain  than  it  really  is.  In  other  words,  the  chance  of  an  event 
is  either  near  zero  or  one.  For  instance,  in  deciding  which  schools  need  additional 
funds,  the  decision  maker  might  choose  to  assess  current  levels  of  needs  and  ignore  the 
uncertainty  about  future  needs.  Of  course,  such  simplifications  are  only  appropriate 
when  using  them  will  make  little  difference  in  the  results  of  the  analysis.  Alternatively, 
the  analyst  may  assume  that  uncertainty  is  the  only  issue  and  that  the  other  values 
and  actions  can  be  addressed  without  the  help  of  analysis. 

Good  analysis  is  about  the  process  not  the  numbers.  One  way  to  analyze  a 
decision  is  for  the  analyst  to  conduct  an  independent  analysis  and  present  the  results 
to  the  decision  maker  in  a  brief  paper.  This  is  usually  not  very  helpful  and  emphasizes 
the  findings  as  opposed  to  the  process.  Decision  makers  are  more  likely  to  accept  an 
analysis  in  which  they  have  actively  participated. 

The  preferred  method  is  to  conduct  decision  analysis  as  a  series  of  increasingly 
more  sophisticated  communications  with  the  decision  maker.  At  each  communication, 
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the  analyst  listens  and  summarizes  the  decision  maker’s  statements.  In  each  step,  the 
problem  is  structured,  and  an  analytical  model  is  created.  Through  these  cycles,  the 
decision  maker  comes  to  certain  determinations,  and  the  analyst  documents  his/her 
conclusions. 

It  is  difficult  to  evaluate  the  effectiveness  of  decision  analysis  because,  often  no 
information  is  available  on  what  might  have  happened  if  decision  makers  had  not 
followed  the  course  of  action  recommended  by  the  analysis.  One  way  to  improve  the 
accuracy  of  analysis  is  to  make  sure  that  the  process  of  analysis  is  followed  accurately. 
Rouse  and  Owen  suggest  asking  the  following  questions  about  decision  analysis  to 
determine  if  it  was  done  accurately  [7]: 

1.  ’’Were  all  realistic  strategies  included?” 

2.  ’’Was  the  appropriate  type  of  model  employed?” 

3.  ’’Were  all  important  outcomes  considered?” 

4.  ’’Was  an  explicit  and  sensible  process  used  to  identify,  select  and  combine  the 
evidence  into  probabilities?” 

5.  ’’Were  utilities  assigned  to  outcomes  conceivable,  and  were  they  obtained  in  a 
methodologically  acceptable  manner?” 

6.  ’’Was  the  potential  impact  of  any  uncertainty  in  the  probability  and  utility 
estimates  thoroughly  and  systematically  evaluated?” 

These  authors  also  point  out  four  serious  limitations  to  decision  analysis  which 
are  important  to  keep  in  mind: 
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1.  ’’Decision  analysis  may  oversimplify  problems  to  the  point  that  they  do  not 
reflect  the  real  concerns  and  accurately  represent  the  perspective  from  which 
the  analysis  is  being  conducted.” 

2.  ’’Available  data  simply  may  be  inadequate  to  support  the  analysis.” 

3.  ’’Utility  assessment,  in  particular  assessment  of  quality  of  life,  may  be  problem¬ 
atic.” 

4.  ’’Outcomes  of  decision  analysis  are  not  agreeable  to  traditional  statistical  anal¬ 
ysis.  Strictly  by  the  principles  of  decision  analysis,  the  preferred  strategy  or 
treatment  is  the  one  that  yields  the  greatest  utility  (or  maximizes  the  occurrence 
of  favorable  outcomes)  no  matter  how  narrow  the  margin  of  improvement.” 

In  the  end,  the  value  of  decision  analysis  (with  all  of  its  limitations)  is  in  the 
eye  of  the  one  who  regards.  If  the  decision  maker  can  understand  and  have  new 
insights  into  a  problem,  and  if  the  problem  and  suggested  course  of  action  can  be 
documented  and  communicated  to  others  more  easily,  a  decision  maker  may  judge 
decision  analysis,  even  imperfect  analysis,  as  useful. 

2.1  Value  Focused  Thinking 

The  methodology  for  this  research  is  a  decision  analysis  technique  called  value 
focused  thinking  (VFT).  Traditional  decision  making  concentrates  on  the  alternatives 
and  their  potential  outcomes.  However,  the  VFT  process  focuses  on  the  values  of 
the  decision  maker  rather  than  the  alternatives  that  are  already  available  at  hand. 
Keeney  states  that  ’’alternatives  are  only  means  to  achieve  objectives”  [23].  ’’Values 


25 


are  the  fundamental  objectives  that  the  decision  seeks  to  achieve,  so  they  should 
be  the  focus  of  analysis”  [22],  ’’This  is  considered  a  proactive  rather  than  reactive 
method  of  examining  the  problem”  [23]. 

Fundamental  objectives  refer  to  the  objectives  underlying  the  essential  reasons 
for  the  problem  being  under  consideration  while  means  objectives  are  regarded  as 
those  whose  fulfillment  will  help  achieve  the  fundamental  objectives.  To  perform  this 
step,  Keeney’s  ’’Why  is  this  important?”  test  is  done.  Each  objective  is  evaluated 
against  this  question,  and  if  an  objective  is  found  to  be  important  because  it  helps 
achieve  another  objective,  it  is  categorized  as  a  means  objective.  Otherwise  it  is  a 
fundamental  objective. 

There  are  two  primary  methods  of  thinking  about  decisions:  alternative  focused 
thinking  and  value  focused  thinking.  The  difference  between  the  two  is  simple.  Alter¬ 
native  focused  thinking  (AFT)  considers  the  available  alternatives  and  subsequently 
compares  them  to  each  other,  while  value-focused  thinking  (VFT)  compares  alter¬ 
natives  to  organizational  values.  As  expressed  by  Keeney,  ’’value  focused  thinking 
implies  that  one  determines  what  is  important  and  subsequently  figures  out  how  to 
get  it”  [22],  While  making  decisions  based  only  on  available  alternatives  gets  the 
job  done,  it  constrains  the  ability  of  an  individual  or  organization  to  achieve  their 
true  values.  Keeney  summarizes  the  fundamental  difference  between  alternatives  and 
values  as  ’’the  values  that  are  fundamentally  important  in  any  decision  situation.  Al¬ 
ternatives  are  relevant  only  because  they  are  the  means  to  achieve  the  values.”  [22], 
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He  continues  to  indicate  that  ’’consequences  are  the  result  of  decisions,  and  the  de¬ 
sirability  of  consequences  is  a  concept  based  on  values”  [22], 

Kirkwood  explains  clearly  the  four  primary  uses  of  the  VFT  process  as  an  im¬ 
portant  tool.  First,  ”it  assists  organizations  in  collecting  appropriate  information”. 
That  is,  it  highlights  what  is  important,  by  means  of  allowing  an  organization  to  focus 
on  collecting  relevant  information  that  is  outside  the  decision  maker’s  concern.  Sec¬ 
ond,  ’’when  there  are  no  previously  present  alternatives,  the  VFT  process  facilitates 
focused  brainstorming  that  leads  to  the  development  of  alternatives  which  address 
values  important  to  a  decision”.  Third,  ’’the  VFT  process  provides  clear  communi¬ 
cation”.  It  demonstrates  to  the  ones  who  hold  a  share  or  interest  what  the  decision 
makers  consider  important.  It  facilitates  the  objective  defense  of  specific  alternative 
selections.  Finally,  ’’the  VFT  process  provides  the  model  for  evaluating,  and  subse¬ 
quently  ranking,  the  alternatives  with  respect  to  the  value  added  to  an  organization 
(or  individual)”  [25]. 

The  process  used  in  this  research  was  the  ten  step  process  shown  in  Figure  2.1. 
In  the  first  step,  the  fundamental  problem  is  identified.  This  helps  to  focus  the  anal¬ 
ysis  on  exactly  what  the  decision  maker  is  trying  to  achieve.  The  value  hierarchy  is 
created  in  the  second  step.  All  of  the  decision  maker’s  values  are  identified  and  then 
organized  into  a  hierarchy.  The  tiers  of  a  value  hierarchy  show  the  relative  importance 
of  the  evaluation  considerations.  The  most  important  values  should  be  in  the  first 
tier;  these  values  are  further  decomposed  into  various  tiers  of  sub-values.  Kirkwood 
emphasizes  that  ’’the  value  hierarchies  should  be  complete,  non- redundant,  decom- 
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posable,  operable  and  relatively  small”  [25].  A  value  hierarchy  contains  evaluation 
considerations,  objectives,  and  evaluation  measures.  In  the  third  step,  the  means  to 
measure  the  lowest  tier  values  are  determined.  The  scale  used  to  score  an  evalua¬ 
tion  measure  can  be  natural  or  constructed  and  direct  or  proxy.  Kirkwood  says  that 
’’profit  in  dollars  is  a  natural  scale  and  gross  national  product  as  a  measure  of  eco¬ 
nomic  well-being  is  a  proxy  scale”  [25].  In  the  fourth  step,  the  decision  maker  creates 
value  functions  for  each  measure.  The  y-axis  will  have  a  range  of  zero  to  one,  and 
the  x-axis  will  be  the  potential  range  of  each  measure.  This  step  not  only  normalizes 
the  measures,  but  also  encourages  the  decision  maker  to  realistically  think  about  the 
measures  and  determine  what  quantities  are  desirable.  In  the  fifth  step,  the  decision 
maker  determines  weights  for  each  value  and  measure  in  the  hierarchy.  In  this  step, 
they  are  identifying  how  important  a  value  is  relative  to  the  other  values  in  the  hier¬ 
archy.  In  the  sixth  step,  alternatives  are  generated.  In  the  seventh  step,  the  scoring 
of  the  alternatives  by  evaluating  each  alternative  is  done  against  the  measures. 

After  creating  the  hierarchy  and  scoring  the  alternatives,  analysis  can  begin  as 
outlined  by  Keeney.  In  the  eighth  step,  deterministic  analysis  is  performed  for  each 
alternative  by  adding  the  weighted  value  of  the  measure  score  to  produce  an  overall 
score.  The  alternatives  with  higher  values  are  preferred  over  those  with  lower  values. 
In  the  ninth  step,  a  sensitivity  analysis  is  performed  to  determine  how  sensitive  the 
alternatives  are  to  changes  in  the  weights  of  the  hierarchy.  For  each  value  and  measure, 
the  weight  is  varied  to  see  how  the  ranking  of  alternatives  changes.  Finally  in  the 
tenth  step,  recommendations  for  the  most  preferred  alternatives  are  made  [22],  The 
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Figure  2.1:  Value  Focused  Thinking  Process 

result  of  this  process  is  identification  of  alternatives  that  reflect  and  fulfill  the  decision 
maker’s  values. 

Kirkwood  states  that  ’’the  value  focused  thinking  helps  to  create  better  alterna¬ 
tives  for  decision  problems”  [25].  Alternative  focused  thinking  is  a  reactive  approach. 
”  A  decision  problem  arises,  and  alternatives  are  generated  to  solve  that  problem 
which  result  in  a  limited  pool  of  alternatives”  as  stated  by  Keeney  [23].  Conversely, 
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for  VFT,  the  fundamental  values  of  the  decision  maker  are  identified  first,  so  actions 
can  be  taken  to  achieve  those  values.  In  Step  6  of  the  ten  step  process,  alternatives 
are  generated  based  on  the  value  hierarchy.  The  result  is  a  pool  of  more  creative 
alternatives  that  better  reflect  the  decision  maker’s  values.  Keeney  expresses  that 
’’the  value  focused  thinking  helps  to  develop  an  enduring  set  of  guiding  principles  for 
an  organization”  [23].  Whether  there  is  a  decision  opportunity  or  not,  it  is  useful  for 
an  organization  to  list  and  organize  its  fundamental  objectives.  ’’For  many,  simply 
listing  their  values  allows  for  more  focused  actions  to  achieve  those  values”  as  stated 
by  Kirkwood  [25]. 

VFT  allows  the  decision  maker  or  the  analyst  the  ability  to  focus  his/her  efforts 
on  assembling  the  right  information  for  the  problem  according  to  what  was  identified 
as  being  important.  VFT  process  is  also  helpful  in  creating  the  suitable  alternatives 
in  situations  where  previously  existing  alternatives  are  not  readily  available.  By  using 
VFT,  all  the  important  considerations  are  taken  into  account  for  all  involved.  This  is 
especially  important  when  the  decision  problem  is  large  enough  to  have  many  people 
involved  in  the  outcome.  To  end  with,  VFT  concentrates  on  determining  the  values 
at  the  core  of  the  decision. 
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III.  Methodology 


3.1  Overview 

This  chapter  describes  how  the  topics  and  methods  from  Chapter  2  were  applied 
to  the  value-focused  thinking  based  individual  versus  group  decision  problem  on  the 
case  study.  The  value-focused  thinking  process  is  detailed  in  creating  the  individual 
and  group  hierarchies  for  the  case  study  question.  While  creation  of  the  individual 
hierarchies  was  not  a  complicated  process,  there  was  difficulty  in  the  group  session 
in  coming  up  with  consensus  over  scoring  the  measures  which  was  already  the  main 
observation  purpose  of  the  research  topic.  Group  and  individual  decision  processes 
have  been  discussed  on  an  interesting  topic  which  every  country’s  Air  Force  faces: 
What  is  the  best  primary  training  aircraft  for  pilot  training?  The  decision  makers  were 
all  seasoned  military  pilots  who  are  from  different  ranks,  ages,  experience  levels,  forces 
and  countries.  The  best  primary  training  aircraft  hierarchies  for  the  individuals  and 
the  group  are  constructed  throughout  the  chapter  as  the  process  steps  are  described. 
After  following  the  steps  dictated  in  VFT  method  through  chapter-3,  evaluating  the 
individual  decisions  versus  group  decision,  will  be  analyzed  in  step-8:  Deterministic 
Analysis  in  chapter-4.  The  discussion  will  be  made  on  whether  the  individual  decisions 
differ  from  the  group  decision  on  the  same  topic  together  with  the  resulting  decision 
and  the  suggestions  will  be  presented  on  the  case  study  topic:  the  best  primary  training 
aircraft  by  the  end  of  chapter-4. 
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3.2  Problem  Identification 


Problem  identification  is  very  important  to  clearly  establishing  the  focus  of  the 
decision  making  process;  it  is  often  the  most  difficult,  and  always  the  most  important 
step.  Primary  flight  training  is  the  most  significant  part  of  the  flight  training  program. 
Some  considerations  for  choosing  a  primary  training  aircraft  might  be  that  it  is  safe 
enough  for  relatively  inexperienced  students  to  fly  solo  in  clear  air  and  in  clouds,  in 
turbulence  and  in  smooth  air,  in  good  and  bad  weather,  at  low  level  and  at  altitude. 
The  aircraft  must  react  safely  to  typical  student  mistakes  such  as  flying  slightly  out 
of  trim  or  flying  with  less  than  perfect  accuracy,  such  as  on  the  downwind  leg  in 
the  circuit,  in  the  critical  final  turn  before  landing,  in  night  flying  or  in  flying  at  low 
altitude.  Due  to  the  importance  of  choosing  the  best  primary  trainer  for  primary  flight 
training,  decision  makers  were  asked:  ’’What  is  important  to  you  in  a  primary  training 
aircraft  in  order  to  choose  the  best  primary  trainer?”  The  decision  makers  play  a  key 
role  in  this  process  since  they  are  typically  the  ones  faced  with  the  requirement  to  solve 
the  problem.  For  this  research,  however,  a  problem  was  initially  identified  through 
the  literature  review  presented  in  the  previous  chapter.  This  made  it  very  important 
to  ensure  that  the  problem  initially  identified  was  identified  correctly.  This  step  was 
further  complicated  by  the  fact  that  making  such  a  decision  for  any  military  force  is 
not  realistic  with  a  single  decision  maker.  To  address  this  decision  maker  problem,  15 
decision  makers,  who  can  expectedly  become  a  potential  member  of  any  procurement 
phase  for  the  forces  they  serve,  were  used  the  values  obtained  from  these  DMs  are 
assumed  to  represent  the  essential  values  held  by  operational  users  of  that  primary 
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training  aircraft.  The  decision  makers  (DMs)  participating  in  this  research  were  2 
Lieutenant  Colonels  from  the  USAF,  5  Majors  from  USAF,  1  Major  from  USMC, 
3  LCDRs  from  USN  and  4  First  Lieutenants  from  TUAF  (Turkish  Air  Force)  with 
different  service  years  and  experience  levels  (from  both  academic  and  operational 
perspective). 

3.3  Step-2:  Create  the  Value  Hierarchy 

Generating  the  values  related  to  the  overall  objective  and  then  organizing  them 
into  a  value  hierarchy  is  the  next  step  after  clearly  identifying  the  problem.  The  sil¬ 
ver  standard  was  the  initial  method  used  for  identifying  the  values  from  the  decision 
makers,  who  are  also  subject  matter  experts.  Parnell  et  al.  [13]  define  three  standards 
for  developing  multiple-objective  value  models:  platinum,  gold,  and  silver  standard. 
A  platinum  standard  process  uses  interviews  with  senior  stakeholders  and  decision 
makers  to  determine  the  objectives.  A  gold  standard  process  determines  the  objec¬ 
tives  from  policy  or  strategic  planning  documents  approved  by  the  decision  makers. 
The  silver  standard  uses  interviews  with  subject-matter  experts  and  stakeholder  rep¬ 
resentatives.  This  project  used  the  silver  standard  method.  As  the  decision  makers 
did  not  consist  of  the  top  level  military  officials  with  the  authority  and  budget  to 
acquire  the  new  primary  trainer,  the  desired  end  result  of  the  decision  process  was  an 
identification  of  the  values  and  preferences  at  the  user  level  that  could  give  a  robust 
beginning  point  for  the  future  acquisition  process. 

The  following  concepts  were  defined  for  each  DM: 
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Decision:  An  allocation  of  resources  whose  change  would  be  costly. 

Decision  Analysis:  Discipline  for  evaluating  complex  alternatives  by  systematically 
examining  decisions,  uncertain  variables  and  preferences  (value,  risk  and  time). 

Purpose  of  DA:  Provides  insight  to  decision  makers  faced  with  hard  problems. 

Structure  of  Strategic  Decisions:  Composes  of  values  (What  do  we  want?),  in¬ 
formation  (What  do  we  know?)  and  alternatives  (What  can  we  do?).  The  key 
idea  is  using  our  strategic  values  to  create  decision  opportunities. 

Change  to  Thinking:  The  classical  thinking  process  (Alternative  Focused  Think¬ 
ing)  starts  with  the  present  alternatives  and  implements  and  evaluation  among 
those  whereas  Value  Focused  Thinking  adds  new  alternatives  to  the  eixsting  at 
hand  in  order  to  cover  all  the  values  of  the  decision  maker  and  then  implements 
the  evaluation  among  those.  It  doesn’t  focus  on  the  alternatives.  It  does  focus 
on  the  values. 

Value  Focused  Thinking  (VFT)  Approach:  Guides  to  strategic  thinking,  facili¬ 
tates  involvement  in  multiple-stakeholder  decisions  and  uncovers  hidden  objec¬ 
tives. 

DA  Applications:  Private  (automotive;  General  Motors,  Ford,  oil  &  gas;  Chevron, 
Phillips  Petroleum,  pharmaceutical;  Eli  Lilly,  R  &  D  portfolios,  etc.),  public  ( 
DOD;  Army,  Air  Force,  Navy,  Intel  Agencies,  DOE;  nuclear  waste,  hazardous 
chemicals,  NASA,  public  utilities,  etc.) 

VFT  methodology:  10-Step  VFT  process.  [33] 


34 


After  they  had  a  clear  idea  of  VFT,  the  DMs  were  asked  the  question:  "If 
you  had  no  limitation  at  all,  what  would  your  objectives  be  in  a  primary  training 
aircraft?”.  They  were  asked  to  answer  the  question,  under  the  circumstances  that 
they  had  no  limitations.  If  they  could  have  anything  they  wanted,  what  would  they 
desire  in  a  primary  training  aircraft?  Also  some  alternative  questions  were  created, 
such  as  ’’What  are  some  problems  or  missing  parts  in  the  current  training  aircraft  that 
they  would  want  corrected  in  future  trainers?” ,  to  help  the  brainstorming.  Each  DM 
provided  his  wish  list  in  a  primary  training  aircraft  (Typically  capable  of  high  subsonic 
speeds,  high-energy  maneuvers,  and  equipped  with  systems  that  simulate  modern 
weapons  and  surveillance  or  more  advanced  training.  Examples  of  such  trainer  aircraft 
include  the  T-38  Talon  (actually  capable  of  supersonic  speeds),  the  BAE  Hawk,  the 
Alpha  Jet  and  the  Aero  L-39,  which  are  used  to  develop  piloting,  navigational  or 
weapon-aiming  skills  in  military  pilots  who  progress  to  training  for  ’’fast  jet”  flying 
and  will  then  progress  to  a  jet  trainer.)  Thus,  a  wish  list  has  been  created  for  every 
individual  decision  maker.  An  individual  value  hierarchy  was  constructed  for  each 
DM. 

The  entire  list  of  the  values  given  by  the  decision  makers  are  listed  below  in 
Table  3.1. 
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Table  3.1:  Entire  List  of  Values 


TOP  TIER  VALUE 

SUBTIER  VALUE 

DESIGN 

User  Friendly  /  Throttle 

User  Friendly  /  Avionics 

Simplicity  /  Standardized  Cockpit 

Simplicity  /  Emergency  Procedures 

Robustness  /  Landing  Gears 

Robustness  /  Airframe  Lifetime 

Ergonomics  /  Cockpit 

Ergonomics  /  Visibility 

Ergonomics  /  Noise  Level 

Ergonomics  /  ECS 

Systems  Complexity 

Systems  Dependency  /  Engine  Start 

Upgradeability 

Styling 

Deactivation  Capability 

PERFORMANCE 

Endurance 

Thrust 

Fuel  Efficiency 

Speed 

Range 

Ceiling 

Max  Take-off  Runway  Length 

Power  Loading 

RELIABILITY 

Mean  Time  Between  Failures  (MTBF) 

MAINTAINABILITY 

Mean  Time  Between  Maintenance  (MTBM) 

Time  Between  Overhaul  (TBO) 

Recording  Capacity 

Maintenance  Specialty  Requirement 

Civilian  Airports  Cross  Service 

SAFETY 

Total  Safety  Incidents  (Class  A,B,C) 

Engine 

Ejection  Seat 

Hook 

Drag  Chute 

Collision  Avoidance  System 

FLYING  QUALITY 

Recoverability 

Stability 

Maneuverability  /  Roll  Rate 

Maneuverability  /  G  Capacity 

Handling  Quality  Rating 

Flight  Path  Stability 

TRAINING  QUALITY 

Instrument  Flight 

Formation  Flight 

Low  Level  Flight 

Aerobatics 

Ground  Handling 

TECHNOLOGY 

Consistency  With  Current 

NAVAIDS 

Comm  System 

Radar 

COST 

Maintenance  Cost 

Aircraft  Price 

SUPPORTING  SYSTEMS 

Synthetic  Training  System 

Debriefing  System 

Life  Support  Materials  /  G-Suit 
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The  initial  layout  of  the  decision  makers’  hierarchies  was  developed  with  the  use 
of  the  silver  standard  and  submitted  to  the  decision  makers  for  their  approval.  The 
first  interviewed  DM:  DM-3  (DMs  are  listed  in  alphabetical  order  based  on  their  last 
names),  has  given  his  wish  list,  which  included  a  user  friendly  aircraft  with  easy-to- 
use  throttle  and  avionics,  the  ability  to  forgive  inexperienced  pilots  with  its  stability, 
low  maintenance,  high  reliability,  safety  and  enough  endurance  to  provide  adequate 
sortie  time  to  be  able  to  teach  the  student  all  the  sortie  requirements.  His  values  were 
organized  in  the  value  hierarchy  below  in  Figure  3.1. 


Figure  3.1:  DM-3  Value  Hierarcy 


From  this  hierarchy,  some  of  the  values  (i.e.  user-friendly,  stability)  were  placed 
under  some  other  values.  Even  though  the  ’’User  Friendly”  is  one  of  his  main  values, 
it  is  placed  under  ”  Design”  in  order  to  make  it  look  more  understandable  for  someone 
who  is  not  familiar  with  those  terms.  ’’Stability”  is  placed  under  ’’Flying  Quality”. 
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Apparently,  when  all  the  DMs  are  taken  into  consideration,  ’’User  Friendly”  is  not  the 
only  value  which  is  under  ’’Design”.  It’s  only  one  out  of  9  values  meant  by  15  DMs 
(User  Friendly,  Simplicity,  Robustness,  Ergonomics,  Systems  Complexity,  Upgrade- 
ability,  Styling  and  Deactivation  Capability),  which  can  be  grouped  under  ’’Design”. 
This  is  a  preference  in  creating  the  hierarchy  to  make  it  look  more  organized. 

After  constructing  all  the  individual  hierarchies  and  determining  the  evaluation 
measures,  the  group  came  together  in  order  to  implement  the  same  process  altogether. 
After  accomplishing  all  the  interviews  with  the  DMs  to  create  their  individual  hier¬ 
archies,  the  group  session  was  held  on  the  13th  of  December,  2007.  The  same  process 
has  been  applied  to  the  group  as  it  was  for  the  individuals.  The  group  was  asked 
to  create  a  wish  list  in  the  first  10  minutes  of  the  session.  The  group’s  values  were 
organized  in  the  value  hierarchy  shown  in  Figure  A.l  in  Appendix- A. 

Though  the  top-tier  values  varied  slightly  among  the  decision  makers’  individual 
hierarchies,  there  were  common  values  within  each  of  them.  Design,  performance, 
reliability,  maintainability,  safety,  flying  quality,  training  quality,  technology,  cost  and 
supporting  systems  were  the  main  and  most  important  values  for  a  primary  training 
aircraft  selected  by  the  decision  makers.  Each  of  these  top  tier  values  in  return 
has  lower  tier  values  to  specify  further  what  the  decision  makers  determined  to  be 
important  in  evaluating  the  primary  training  aircraft,  leading  to  mutually  exclusive, 
collectively  exhaustive  measures.  A  definition  of  each  value  has  been  presented  in 
Appendix-C  in  detail. 
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Design  embraces  the  concept  of  bringing  together  all  of  the  aspects  of  airframe 
and  gear  robustness,  simplicity  of  operational  procedures,  ergonomics,  systems  com¬ 
plexity  and  dependency  on  other  systems,  upgradeability  of  the  systems,  style  and 
user  friendliness.  Design,  as  a  top-tier  value,  has  been  covered  by  15  out  of  15  DMs. 

Performance  comprises  endurance,  thrust,  speed,  range,  service  ceiling,  max¬ 
imum  take-off  runway  length,  fuel  efficiency  and  power  loading  which  may  also  be 
known  as  power/weight  ratio.  Performance,  as  a  top-tier  value,  has  been  covered  by 
13  out  of  15  DMs. 

Reliability,  as  the  ability  of  a  system  or  component  to  perform  its  required 
functions  under  stated  conditions  for  a  specified  period  of  time,  is  measured  by  MTBF 
(Mean  Time  Between  Failures)  which  assumes  that  a  system  is  renewed  or  fixed  after 
each  failure  and  then  returned  to  service  immediately  after  failure.  Reliability,  as  a 
top-tier  value,  has  been  covered  by  6  out  of  15  DMs. 

Maintainability  measures  the  ease  and  speed  with  which  a  system  can  be  re¬ 
stored  to  operational  status  after  a  failure  occurs.  It  is  quantified  mainly  in  terms 
of  MTBM  (Mean  Time  Between  Maintenance),  but  also  covers  TBO  (Time  Between 
Overhaul),  recording  capacity  of  the  system,  maintenance  specialty  requirements,  and 
civilian  airports  cross-service  as  the  other  components  in  the  sub-tier  contributing  to 
maintainability.  Maintainability,  as  a  top-tier  value,  has  been  covered  by  9  out  of  15 
DMs. 
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Safety  comprises  the  total  safety  incident  history  (class  A  &  B  &  C),  number  of 
engines,  ejection  seat  availability  and  quality,  collision  avoidance  system  availability, 
hook  and  drag  chute  systems  availability  in  the  second  tier.  Safety,  as  a  top-tier  value, 
has  been  covered  by  10  out  of  15  DMs. 

Flying  quality  includes  stability,  recoverability,  maneuverability  (roll  rate  and 
g-capacity),  handling  quality  rating  (Cooper- Harper  score),  and  flight  path  stability. 
Flying  quality,  as  a  top-tier  value,  has  been  covered  by  15  out  of  15  DMs. 

Training  quality  includes  instrument,  formation,  low-level,  aerobatics  flight  train¬ 
ing  and  ground  handling  quality.  Training  quality,  as  a  top-tier  value,  has  been  covered 
by  5  out  of  15  DMs. 

Technology  includes  NAVA1DS,  communication  system,  radar  (if  available)  and 
consistency  level  of  those  systems  with  the  current  systems  in  use.  Technology,  as  a 
top-tier  value,  has  been  covered  by  10  out  of  15  DMs. 

Cost  includes  the  maintenance  cost  and  the  aircraft  price.  Cost,  as  a  top-tier 
value,  has  been  covered  by  4  out  of  15  DMs.  Supporting  systems  include  the  synthetic 
training  system  and  debriefing  system  life  support  materials  used  on  board  (i.e.  g- 
suit).  Supporting  systems,  as  a  top-tier  value,  has  been  covered  by  5  out  of  15  DMs. 

All  the  necessary,  important  properties  of  a  good  value  model  were  taken  into 
consideration  while  constructing  the  value  hierarchies.  For  instance,  every  value  is 
covered  by  a  correctly  identified  measure,  and  each  value  is  connected  to  the  next 
higher  tier  in  order  for  each  hierarchy  to  be  complete.  Overlapping  values  have  been 
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eliminated  to  avoid  double-counting.  Eventually,  the  study  involved  all  the  properties 
that  proved  to  be  important  to  the  hierarchy  which  must  be  as  small  as  possible, 
understandable  and  explainable  for  each  DM.  The  fundamental  and  means  objectives 
were  determined  in  the  first  part  of  constructing  the  hierarchy.  The  hierarchy  started 
with  the  fundamental  objectives  and  moved  downward  into  the  means  objectives. 
Moving  down  the  hierarchy  continued  until  there  was  nothing  left  to  ask  ’’why  is  it 
important?”  The  decision  was  covered  completely  and  accurately  with  many  different 
tier  levels  of  the  individuals’  hierarchies. 

3.4  Step-3:  Develop  Evaluation  Measures 

The  next  step  was  to  develop  the  evaluation  measures  for  each  lowest-level 
objective  after  completing  the  value  hierarchy.  The  measures  provide  the  VFT  model 
the  capability  to  evaluate  an  alternative  in  a  quantitatively  objective  manner.  Hence, 
the  measure  value  functions  must  be  clearly  defined  to  eliminate  variability  as  much  as 
possible  throughout  the  evaluation  process.  These  measures  were  determined  to  best 
represent  how  to  achieve  the  values  in  the  hierarchy  while  maintaining  the  integrity 
of  independence  and  non-redundancy.  The  decision  makers  were  actively  involved  in 
the  development  of  the  measures.  DM-3  (as  the  DM  of  the  hierarchy  shown  in  Figure 
3.1)  has  offered  his  own  measures  for  the  user  friendly  throttle  value  under  ’’Design” 
as  well  as  the  avionics.  Other  measures,  searched  and  found  through  the  literature, 
were  presented  to  him  as  ones  that  he  might  consider  to  scale  his  values,  and  these 


41 


were  approved  by  him.  Since  all  the  measures  were  set  by  the  end  of  the  individual 
interviews,  everything  was  ready  when  it  came  to  the  group  hierarchy  measures. 

Each  value  was  measured  either  quantitatively  or  qualitatively.  These  evaluation 
measures  were  based  on  a  combination  of  two  different  classifications.  The  classifica¬ 
tions  are  either  natural  or  constructed,  and  either  direct  or  proxy.  While  creating  the 
measures,  the  goal  was  to  achieve  as  many  natural- direct  or  constructed-direct  scales 
as  possible  and  as  few  natural-proxy  or  constructed-proxy  scales  as  possible  for  ease 
of  use  and  understanding.  Natural-direct  is  the  best  to  work  with  because  it  gives  a 
clear  picture  of  how  a  value  is  measured  and  does  not  need  any  subject  matter  experts 
to  construct  scales  or  proxy  measures.  However,  the  natural,  direct  characteristics 
could  not  be  met  all  the  time  clue  to  the  specific  nature  of  the  measures  listed  by  some 
of  the  decision  makers.  All  the  evaluation  measures,  types  and  SDVFs  are  listed  in 
Table  3.2. 
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Tabic  3.2:  Subtier  Values-Evaluation  Measures  List 


SUBTIER  VALUE 

Measure 

SDVF 

User  Friendly  /  Throttle 

Throttle  Star  Value 

Categorical 

User  Friendly  /  Avionics 

Avionics  Scale 

Categorical 

Simplicity  /  Standardized  Cockpit 

Standardization  Level 

Categorical 

Simplicity  /  Emergency  Procedures 

Average  Number  of  Steps  in  Boldfaces 

Categorical 

Robustness  /  Landing  Gears 

Failure  Rate  PerlOOK  F/H 

Categorical 

Robustness  /  Airframe  Lifetime 

Years 

Continuous 

Ergonomics  /  Cockpit 

Cockpit  Star  Value 

Categorical 

Ergonomics  /  Visibility 

Visibility  Star  Value 

Categorical 

Ergonomics  /  Noise  Level 

dB 

Continuous 

Ergonomics  /  ECS 

ECS  Star  Value 

Categorical 

Systems  Complexity 

Complexity  Level 

Categorical 

Systems  Dependency  /  Engine  Start 

Engine  Start  Dependency  Level 

Categorical 

Upgradeability 

Upgradeability  Level 

Categorical 

Styling 

Styling  Star  Value 

Categorical 

Deactivation  Capability 

Deactivation  Feature  Star  Value 

Categorical 

Endurance 

Minutes 

Continuous 

Thrust 

Lbs. 

Continuous 

Fuel  Efficiency 

Gallons  Per  F/H 

Continuous 

Speed 

Knots 

Continuous 

Range 

Miles 

Continuous 

Ceiling 

Feet 

Continuous 

Max  Take-off  Runway  Length 

Feet 

Continuous 

Power  Loading 

Power/Weight  Ratio 

Continuous 

Mean  Time  Between  Failures  (MTBF) 

Hours 

Continuous 

Mean  Time  Between  Maintenance  (MTBM) 

Hours 

Continuous 

Time  Between  Overhaul  (TBO) 

Hours 

Continuous 

Recording  Capacity 

Recording  Capacity  Level 

Categorical 

Maintenance  Specialty  Requirement 

Specialty  Certificate  Requirement  Level 

Categorical 

Civilian  Airports  Cross  Service 

Cross  Service  Support  Level 

Categorical 

Total  Safety  Incidents  (Class  A,B,C) 

Total  Number  PerlOOK  F/H 

Categorical 

Engine 

Number  of  Engines 

hH  EBBfi  i  «?3 1  WHHI 

Ejection  Seat 

Ejection  Seat  Star  Value 

Categorical 

Hook 

Hook  Speed  Limit 

Continuous 

Drag  Chute 

Drag  Chute  Speed  limit 

Continuous 

Collision  Avoidance  System 

Avoidance  System  Star  value 

Categorical 

Recoverability 

Stall  Warning  Before  Stall  Margin_Knots 

Categorical 

Stability 

Degrees  of  Stability 

Categorical 

Maneuverability  /  Roll  Rate 

Degrees  Per  Second 

Continuous 

Maneuverability  /  G_Capacity 

Gs 

Categorical 

Handling  Quality  Rating 

Cooper  Harper  Score 

Categorical 

Flight  Path  Stability 

Degrees  Per  Knot 

Continuous 

Instrument  Flight 

Instrument  Flight  Quality  Level 

Categorical 

Formation  Flight 

Formation  Flight  Quality  Level 

Categorical 

Low  Level  Flight 

Low  Level  Quality  Level 

Categorical 

Aerobatics 

Aerobatics  Quality  Level 

Categorical 

Ground  Handling 

Ground  Handling  Quality  Level 

Categorical 

Consistency  With  Current 

Currency  Scale 

Categorical 

NAVAIDS 

NAVAIDS  Scale 

Categorical 

Comm  System 

Comm  System  Quality  Level 

Categorical 

Radar 

Radar  Capability  Level 

Categorical 

Maintenance  Cost 

Hundred  Dollars 

Continuous 

Aircraft  Price 

Million  Dollars 

Continuous 

Synthetic  Training  System 

Synthetic  T rainer  Star  Value 

Categorical 

Debriefing  System 

Debriefing  System  Star  Value 

Categorical 

Life  Support  Materials  /  G-Suit 

G-Suit  Quality  Level 

Categorical 
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The  definitions  for  each  of  the  measures  are  explained  in  Appendix-C.  The 
decision  makers  determined  the  types  of  scales  to  be  used  along  with  upper  and  lower 
bounds.  The  detailed  list  of  all  evaluation  measures  and  the  scales  corresponding  to 
those  measures  under  the  toptier  value  ’’Design”  are  presented  in  Table  3.3.  The  rest 
of  the  evaluation  measure  scores  for  the  other  top  tier  values  are  shown  in  tables  in 
Appendix-A. 

3.5  Step-4-'  Create  Value  Functions 

The  next  step  was  to  create  the  value  functions.  The  purpose  of  creating  the 
value  functions  was  to  be  able  to  convert  the  actual  scores  assigned  to  the  measures 
to  a  corresponding  value  which  represents  the  preference  of  the  decision  maker.  This 
is  determined  by  developing  single  dimensional  value  functions  (SDVF),  which  are 
mathematical  translations  of  the  measures  of  the  values  for  each  decision  maker.  In 
Value  Focused  Thinking,  it  is  important  to  take  into  account  every  individual  decision 
maker’s  values  and  to  agree  on  a  specific  set  of  evaluation  measures,  single-dimensional 
value  functions  ,  and  both  local  and  global  weights. 

The  x-axis  of  the  SDVFs  were  determined  in  the  previous  step;  this  step  will 
determine  the  corresponding  y-axis  values  for  each  category  element  within  its  re¬ 
spective  measure.  The  y-axis  will  always  range  from  a  value  of  0  (least  preferred) 
to  a  value  of  1  (most  preferred)  to  represent  the  full  range  of  the  decision  maker’s 
value.  The  upper  and  lower  bounds  of  the  measures  are  the  equivalent  zero  (lower) 
and  one  (upper)  values  on  the  SDVFs.  This  step  converts  the  qualitative  nature  of 
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Table  3.3:  Design-Evaluation  Measures  &  Scores  List 


User  Friendly  /  Throttle 

Throttle  Star  Value 

1 .  Multilever  Engine  Control  (MLEC) 

2.  Mechanical  Single  Lever  Power  Control  (SLPC) 

3.  Digital  Single  Lever  Power  Control  (SLPC) 

4.  Combined  (Mechanical+  Digital)  SLPC 

User  Friendly  /  Avionics 

Avionics  Scale 

1 .  No  Avionics 

2.  Basic  Flight  Instruments  w/  No  Nav  Sys 

3.  Navigation  System  w /  Manual  Align 

4.  Navigation  System  w /  Auto  Align 

5.  Computer  Driven  Nav  Management  System 

6.  Integrated  Avionics  System 

Simplicity  /  Standardized  Cockpit 

Standardization  Level 

1 .  Basic  6-Pack  Design 

2.  Common  Nav  /  Common  Comm  System 

3.  Integrated  Cockpit  (Pri  Inst  +  Essen  Nav  +  Essen  Comm) 

4.  Standardized  Cockpit 

Simplicity  /  Emergency  Procedures 

Average  Number  of  Steps  in  Boldfaces 

#  steps 

Robustness  /  Landing  Gears 

Failure  Rate  PerlOOK  F/H 

#  failures 

Robustness  /  Airframe  Lifetime 

Flight  Hours 

#  flight  hours 

Ergonomics  /  Cockpit 

Cockpit  Star  Value 

1 . 4-Seat  Cockpit 

2.  Side-by-Side 

3.  Tandem 

4.  Stepped  Tandem 

Ergonomics  /  Visibility 

Visibility  Star  Value 

1 .  Binocular  Cockpit  Visibility 

2.  Clam-Shell  Type  Canopy  +  Body-view  Camera 

3.  Shoulder  Level  Canopy 

4.  Bubble  Canopy 

5.  Bubble  Canopy  With  Transparency 

Ergonomics  /  Noise  Level 

dB 

Ergonomics  /  ECS 

ECS  Star  Value 

1.  No  ECS 

2.  In-Flight  Operated  ECS  System 

3.  50%  Ground  Efficient  +  In-Flight  ECS  System 

4.  Full  Ground  +  In-Flight  ECS  System 

5.  Upgraded  ECS  to  Prevent  Ice  Accumulation,  Vaporized  Air 

Systems  Complexity 

Complexity  Level 

1.  Uncomplicated 

2.  Low/Moderate 

3.  Moderate 

4.  Moderate/High 

5.  High 

Systems  Dependency  /  Engine  Start 

Engine  Start  Dependency  Level 

1 .  APU  Start 

2.  1st_Engine  -  APU  Start  2nd_Engine-By  Means  of  Other  Engine  Flow 

3.  Battery-Power  Start 

Upgradeability 

Upgradeability  Level 

1 .  No  Upgradeability 

2.  Software  Upgradeability 

3.  Modular  Design  -  Improved 

4.  Modular  Open  System 

5.  Built-in  Upgradeability 

Styling 

Styling  Star  Value 

1 .  Round  Shape  Fuselage  with  Straight  Low  Wing 

2.  Bottle  Shape  Body  Type 

3.  Delta  Shape  Body  Type 

4.  Sharp  Triangular  Shape  Body  Type 

Deactivation  Capability 

Deactivation  Feature  Star  Value 

1 .  No  Deactivation  Capability 

2.  Intercom/Radio  System  Deactivation  Capability 

3.  MFD  Modes  Deactivation  Capability 

4.  Stall  Warning  System  Deactivation  Capability 

5.  NAVAIDS  Deactivation  Capability 

the  evaluation  process  into  quantitative  data.  This  allows  the  objective  analysis  to 
be  conducted  later  in  the  modeling  process. 

The  SDVFs  used  in  this  model  are  either  discrete  or  continuous.  The  discrete 
SDVFs  are  categorical,  meaning  they  have  a  finite  number  of  levels,  and  need  to  be 
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represented  categorically.  The  continuous  SDVFs,  which  can  be  either  linear,  piece- 
wise  linear,  or  exponential,  have  an  infinite  number  of  possible  levels.  The  continuous 
SDVFs  are  either  increasing  functions,  having  positive  slopes,  or  decreasing  functions, 
having  negative  slopes.  SDVFs  put  these  values  into  the  same  units  to  allow  weighting 
to  be  applied  correctly. 

The  SDVFs  for  each  measure  must  be  monotonically  increasing,  having  posi¬ 
tive  slopes,  or  decreasing,  having  negative  slopes.  A  monotonically  increasing  value 
function  is  one  in  which  higher  values  on  a  measure  are  preferred  by  the  decision 
maker.  Similarly,  monotonically  decreasing  functions  are  those  for  which  lower  values 
on  evaluation  measures  are  preferred.  In  the  case  of  continuous  functions,  increasing 
functions  have  positive  slopes,  and  decreasing  functions  have  negative  slopes.  Figure 
3.2)  shows  the  discrete,  monotonically  increasing  SDVF  for  the  ’'Throttle  Star  Value” 
measure  in  DM-3’s  hierarchy  as  an  example.  DM-3’s  other  8  SDVFs  for  the  remaining 
measures  are  included  under  figures  in  Appendix-A. 

The  decision  makers  chose  a  discrete  or  continuous  scale  for  each  evaluation 
measure.  For  those  measures  that  were  evaluated  on  a  discrete  scale,  categories  were 
determined  and  given  an  associated  value  by  the  decision  makers.  If  a  measure  was 
determined  to  be  continuous,  the  decision  makers  were  asked  to  provide  an  upper  and 
lower  bound  representing  the  best  and  worst  possible  scores.  Some  decision  makers 
chose  to  use  linear  while  some  others  chose  exponential  functions  for  the  continuous 
functions  based  on  their  inclination  to  either  the  most  or  the  least  preferred  side.  The 
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Figure  3.2:  SDVF  for  Throttle  Star  Value 

decision  makers  based  the  reference  points  and  categories  for  the  value  functions  on 
their  personal  knowledge  and  experience. 

The  process  of  creating  value  functions  was  difficult  and  took  several  iterations 
due  to  its  subjective  nature.  The  decision  makers  were  inclined  to  assess  different 
categories  within  the  same  measure  as  the  same  value.  After  an  explanation  that  an 
exact  value  rating  for  two  different  categories  within  the  same  measure  is  essentially 
equating  those  categories,  the  decision  maker  quickly  changed  the  preference  values  to 
ensure  a  differentiation  between  the  categories.  The  group  session  was  easier  from  this 
point,  since  all  the  decision  makers  had  a  clear  idea  of  the  exact  value  ratings  for  two 
different  categories  from  their  individual  session.  On  the  other  hand,  the  difficulty  of 
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the  group  session  was  in  coming  up  with  consensus  over  scoring  the  measures,  which 
was  already  the  main  observation  purpose  of  the  whole  research  topic. 

VFT  as  a  method  of  DA  has  been  applied  step  by  step  in  chapter-3  in  order 
to  acquire  the  DMs’  values,  evaluation  measures  and  resulting  scores,  which  were 
then  used  as  the  empirical  data  that  yields  direct  information  on  the  acceptability  of 
the  null  hypothesis  that  there  is  no  concordance  between  group  decision  making  and 
individual  decision  making,  to  run  the  analysis  for  evaluating  the  individual  decisions 
versus  group  decision.  Our  decision  abut  the  meaning  of  the  data  would  lead  us  to 
retain,  revise  or  reject  the  hypothesis.  The  analysis  has  been  conducted  in  chapter-4 
in  order  to  determine  the  acceptability  of  hypothesis.  By  the  end  of  chapter-4,  the 
discussion  has  been  made  on  whether  the  individual  decisions  differs  that  much  from 
the  group  decision  on  the  same  topic.  The  resulting  decision  and  the  suggestions  on 
the  case  study  topic:  the  best  primary  training  aircraft  have  also  been  expressed  by 
the  end  of  chapter-4. 

After  the  DMs  provided  their  values,  evaluation  measures  and  ranges  for  the 
corresponding  values  were  constructed.  These  ranges,  increments,  and  values  were  all 
subjective  and  based  on  the  DM’s  individual  preference.  For  the  other  measures,  value 
increments  were  used,  and  the  DM  was  asked  which  increments  were  more  important 
than  others.  In  constructing  value  increments,  the  least  preferred  increment  is  scored 
at  level  k,  and  the  rest  of  the  increments  are  scored  based  on  the  least  preferred  (i.e. 
two  times  more  important,  equal  importance).  Each  k  was  added  to  the  next  and 
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set  equal  to  one.  Knowing  this  information,  k  can  be  computed  and  a  value  can  be 
placed  on  each  increment. 

3.6  Step-5:  Weight  the  Value  Hierarchy 

After  the  SDVFs  have  been  created,  the  DMs  were  interviewed  one  last  time 
individually  for  weighting  the  value  increments  and  the  measures  within  the  hierarchy 
to  score  the  different  alternatives.  The  purpose  of  weighting  the  hierarchy  is  to  apply 
priorities  to  the  evaluation  measures  that  reflect  the  importance  of  each  value  to  the 
decision  maker.  The  method  of  swing  weighting  was  used  to  help  the  DM  determine 
which  value  increments  were  more  important  than  the  others. 

Local  and/or  global  weighting  are  two  ways  to  look  at  this  prioritization  process. 
The  local  weights  are  determined  by  examining  only  the  values  within  the  same  tier  of 
a  branch;  the  local  weights  must  sum  to  one.  The  global  weights  display  how  much  a 
particular  value  contributes  to  the  overall  value  of  an  alternative.  The  swing  weighting 
(or  the  value  increment  procedure)  was  used  for  this  analysis.  The  swing  weighting 
method  begins  with  the  evaluation  measures  placed  from  the  least  to  greatest  value. 
Each  measure  is  then  represented  as  a  multiple  of  the  least  important  value  measure. 
All  weights  are  then  summed  to  one,  and  the  resulting  equation  is  solved  for  the 
weight  of  the  least  valued  measure. 

In  this  research,  the  decision  makers  determined  the  weights  of  the  values  and 
measures  using  a  top-down  approach.  Once  they  agreed  upon  the  weight  values, 
the  local  weights  were  used  to  calculate  the  global  weights.  The  local  and  global 


49 


weights  were  applied  to  the  value  hierarchies  in  their  entirety.  The  weightings  for  each 
individual  and  group  hierarchy  were  determined  using  the  swing  weight  procedure. 

The  measure  ’’Ejection  Seat  Star  Value”  of  the  value  ’’Ejection  Seat”  under  the 
top-tier  value  of  ”  Safety”  is  the  most  significant  measure  in  global  weighting,  which 
has  been  weighted  globally  by  3  decision  makers  with  the  highest  number.  This 
is  to  be  expected  since  safety  is  always  going  to  be  a  significant  factor  in  training 
relatively  unexperienced  pilots  or  pilot  candidates.  The  measures  ’’Total  Number  of 
Safety  Incidents  per  100K  F/H”  under  the  top-tier  value  of  ’’Safety”  and  ’’Endurance 
in  Minutes”  under  the  top-tier  value  of  ’’Performance”,  ”MTBF  in  Flight  Hours” 
under  the  top-tier  value  of  ’’Reliability”,  and  ’’Synthetic  Trainer  System  Star  Value” 
under  the  top-tier  value  of  ’’Supporting  Systems”  are  the  second  most  significant 
measures  in  global  weighting,  which  has  been  outweighted  globally  by  2  decision 
makers.  Other  than  those  measures,  ’’Aerobatics  Quality  Level”  under  the  top-tier 
value  of  ’’Training  Quality”,  ”MTBM  in  Flight  Hours”  under  the  top-tier  value  of 
’’Maintainability”,  "Maintenance  Cost  in  Hundred  Dollars”  under  the  top-tier  value 
of  ”  Cost” ,  ”  Debriefing  System  Star  Value”  under  the  top-tier  value  of  ”  Supporting 
Systems”,  ’’Cockpit  Star  Value”  of  the  value  ’’Ergonomics”  under  the  top-tier  value  of 
’’Design”,  ’’Roll  Rate  in  Degrees  per  Second”  of  the  value  ’’Maneuverability”  under 
the  top-tier  value  of  ’’Flight  Quality”,  ’’Instrument  Flight  Quality  Level”  under  the 
top-tier  value  of  ’’Training  Quality”,  and  ’’Fuel  Efficiency  in  Gallons  per  Flight  Hour” 
under  the  top-tier  value  of  ’’Performance”  are  the  other  significant  measures  in  global 
weighting,  which  has  been  outweighted  globally  by  1  decision  maker.  These  are  the 
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measures  that  the  decision  makers  consider  the  most  important  in  a  primary  training 
aircraft. 

These  measures  can  be  considered  the  significant  attributes.  Every  competing 
primary  training  aircraft  is  expected  to  have  high  scores  within  these  measures  as 
they  represent  a  large  portion  of  what  the  decision  maker  values.  If  the  competition 
is  strong,  it  would  not  be  unusual  for  several  alternatives  to  be  rated  evenly  based 
on  these  measures.  The  other  evaluation  measures  do  not  hold  significantly  high 
global  weights;  however,  their  importance  is  still  significant.  These  measures  are  still 
considered  necessary  by  the  decision  makers;  they  just  happen  to  not  be  rated  as 
important  as  the  top  measures.  What  makes  these  other  measures  significant  is  that 
they  can  help  to  differentiate  between  highly  competitive  alternatives.  If  two  or  three 
alternatives  score  evenly  throughout  the  outweighting  measures,  those  other  measures 
will  allow  selection  of  the  top  alternative. 

3.7  Step-6:  Alternative  Generation 

After  weighting  the  hierarchy,  the  next  step  was  to  generate  the  alternatives. 
Real  world  alternatives  could  change  the  results  either  way  according  to  the  rec¬ 
ommendations  in  the  conclusion  part  of  the  study  ”  Evaluating  the  need  for  Group 
Decision  Making  versus  Individual  Decision  Making  in  Value-Focused  Thinking”  [32], 
The  real  primary  training  aircraft  alternatives,  not  only  the  current  ones  in  use  or 
future  candidates,  but  also  the  old  alternatives  which  have  previously  been  used  as  a 
primary  trainer.  T-34C  (Hawker-Beechcraft),  T-37B  (Cessna),  T-35  Pillan  (ENAER), 
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KT-1C  (KAI),  EMB.312  (Embraer),  T-6A  Texan  II  (Hawker-Beechcraft)  have  been 
selected  as  the  alternatives.  The  required  data  have  been  gathered  from  open  sources 
(i.e.  internet,  Jane’s)  and  companies’  official  representatives  who  are  authorized  to 
disclose  the  information  relating  to  their  aircraft.  All  the  data  has  been  inserted  into 
the  measures-alternatives  table,  showing  the  primary  training  aircraft  alternatives 
considered  for  evaluation.  Below  in  Table  3.4  is  the  part  of  this  data  for  the  T-6A 
Texan  II  aircraft.  The  detailed  list  of  data  for  all  the  alternative  aircrafts,  including 
the  sources  where  the  data  came  from  is  presented  in  Appendix-A. 

3.8  Step-7:  Alternative  Scoring 

The  final  step  before  conducting  analysis  was  to  score  the  alternatives.  As  stated 
in  the  previous  step,  the  required  data  was  obtained  through  several  sources.  This 
data  was  used  to  score  each  primary  trainer  according  to  the  single  dimension  value 
functions  developed  for  each  measure  in  the  hierarchy  of  each  DM  and  group.  Below 
is  DM-3’s  alternative  scoring  as  a  complete  list  in  Table  3.5  and  summary  list  in  Table 
3.6. 
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Table  3.4:  T-6A  Texan  II  Values-Measures-Scales  List 


VALUE 

MEASURE 

SCALE 

DESIGN 

User  Friendly  /  Throttle 

User  Friendly  /  Avionics 

Simplicity  /  Standardized  Cockpit 
Simplicity  /  Emergency  Procedures 
Robustness  /  Landing  Gears 

Robustness  /  Airframe  Lifetime 
Ergonomics  /  Cockpit 

Ergonomics  /  Visibility 

Ergonomics  /  Noise  Level 

Ergonomics  /  ECS 

Systems  Complexity 

Systems  Dependency  /  Engine  Start 
Upgradeability 

Styling 

Deactivation  Capability 

Throttle  Star  Value 

Avionics  Scale 

Standardization  Level 

Average  Number  of  Steps  in  Boldfaces 
Failure  Rate  Per  100K  F/H 

Years 

Cockpit  Star  Value 

Visibility  Star  Value 
dB 

ECS  Star  Value 

Complexity  Level 

Engine  Start  Dependency  Level 
Upgradeability  Level 

Styling  Star  Value 

Deactivation  Feature  Star  Value 

Digital  SLPC 

Integrated  Avionics  System 

Advanced  Cockpit  Layout 

2 

2 

37  Years  (18,720  Flight  Hours) 

Tandem 

Clam-shell  Type  Canopy 

90  dB 

Upgraded  ECS  to  Prevent  Ice  Accumulation, 

Vaporized  Air 

Moderate 

Battery  Power  Start 

Built-in  Upgradeability 

Round  Shape  Fuselage  with  Straight  Low  Wing 

MFD  Modes  Deactivation  Capability 

PERFORMANCE 

Endurance 

Minutes 

180  min. 

Thrust 

Lbs. 

3400  lbs. 

Fuel  Efficiency 

Gallons  Per  F/H 

55  gallons/FH  (400  Ibs/FH) 

Speed 

Knots 

350K 

Range 

Miles 

850  NM 

Ceiling 

Feet 

31000  ft. 

Max  Take-off  Runway  Length 

Feet 

1435  ft. 

Power  Loading 

Power/Weight  Ratio 

0.523  (3400lbs/6500  pounds)3.60  kg/kW  (5.91  Ib/shf 

RELIABILITY 

Mean  Time  Between  Failures  (MTBF) 

Hours 

25  FH 

MAINTAINABILITY 

Mean  Time  Between  Maintenance  (MTB 

Hours 

10  FH 

Time  Between  Overhaul  (TBO) 

Hours 

4,500  Hour  TBO  for  Engine 

Recording  Capacity 

Recording  Capacity  Level 

FDR 

Maintenance  Specialty  Requirement 

Specialty  Certificate  Requirement  Level 

No  Specialty-Certificate  Maintenance  Requirement 

Civilian  Airports  Cross  Service 

Cross  Service  Support  Level 

Stage-A  Cross  Serviceable 

SAFETY 

Total  Safety  Incidents  (Class  A,B,C) 

Total  Number  Per  100K  F/H 

21  (3  Class-A  over  800K  F/H  ;  Class-B,C  - 18) 

Engine 

Number  of  Engines 

1 

Ejection  Seat 

Ejection  Seat  Star  Value 

Zero/Zero  Ejection  Seat 

Hook 

Hook  Speed  Limit 

N/A 

Drag  Chute 

Drag  Chute  Speed  limit 

N/A 

Collision  Avoidance  System 

Avoidance  System  Star  value 

TCAS 

FLYING  QUALITY 

Recoverability 

Stall  Warning  Before  Stall  Margin  Knots 

5  K 

Stability 

Degrees  of  Stability 

Normally  (positively)  stable 

Maneuverability  /  Roll  Rate 

Degrees  Per  Second 

260  degrees/sec 

Maneuverability  /  GCapacity 

Gs 

+7  /  -3.5 

Handling  Quality  Rating 

Cooper  Harper  Score 

2 

Flight  Path  Stability 

Degrees  Per  Knot 

0.06  (Level-1 ; 

TRAINING  QUALITY 

Instrument  Flight 

Instrument  Flight  Quality  Level 

Radio  Instrument  Flight  Training 

Formation  Flight 

Formation  Flight  Quality  Level 

Cruise  +  IFR  Formation  Flight  Training 

Low  Level  Flight 

Low  Level  Quality  Level 

AGL-500'  /  300K 

Aerobatics 

Aerobatics  Quality  Level 

Chndll+Hmmrhd+Loop+lmmlmn+Cbn8+Spin 

Ground  Handling 

Ground  Handling  Quality  Level 

Steering  With  ON/OFF  Buttor 

TECHNOLOGY 

Consistency  With  Current 

Currency  Scale 

Adv  Inst  Panel  +  Adv  Nav  Sys 

NAVAIDS 

NAVAIDS  Scale 

GPS+TACAN+ILS+INS 

Comm  System 

Comm  System  Quality  Level 

UHF+VHF 

Radar 

Radar  Capability  Level 

No  Radar  Capability 

COST 

Maintenance  Cost 

Hundred  Dollars 

$  198  (2.06  MMH/FH) 

Aircraft  Price 

Million  Dollars 

$  4.27  M 

SUPPORTING  SYSTEMS 

Synthetic  Training  System 

Synthetic  Trainer  Star  Value 

Full  Simulator 

Debriefing  System 

Debriefing  System  Star  Value 

Digital  Video  Debriefing  System  (DVDS) 

Life  Support  Materials  /  G-Suit 

G-Suit  Quality  Level 

Standart  Anti-G  Sui 
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Table  3.5:  DM-3  Alternatives  Scoring  Complete  List 


ALTERNATIVE  SCORES 

WEIGHTED  SCORES 

VALUE 

MEASURE 

Global  Weight 

T-34C 

T-37B 

T-6A 

EMB.312 

KT-1C 

T-35 

T-34C 

T-37B 

T-6A 

EMB.312 

KT-1C 

T-35 

DESIGN 

PERFORMANCE 

RELIABILITY 

MAINTAINABILITY 

SAFETY 

FLYING  QUALITY 

TRAINING  QUALITY 

TECHNOLOGY 

COST 

User  Friendly  /  Throttle 

Throttle  Star  Value 

0,0860 

0 

0,66 

0,83 

0,66 

1 

0,66 

0,0000 

0,0568 

0,0714 

0,0568 

0,0860 

0,0568 

User  Friendly  /  Avionics 

Avionics  Scale 

0,1290 

0,7 

0,7 

1 

1 

1 

0,7 

0,0903 

0,0903 

0,1290 

0,1290 

0,1290 

0,0903 

Simplicity  /  Standardized  Cockpit 

Standardization  Level 

Simplicity  /  Emergency  Procedures 

Average  Number  of  Steps  in  Boldfaces 

Robustness  /  Landing  Gears 

Failure  Rate  Per  100K  F/H 

Robustness  /  Airframe  Lifetime 

Years 

Ergonomics  /  Cockpit 

Cockpit  Star  Value 

Ergonomics  /  Visibility 

Visibility  Star  Value 

Ergonomics  /  Noise  Level 

dB 

ergonomics  /  ECS 

ECS  Star  Value 

Systems  Complexity 

Complexity  Level 

Systems  Dependency  /  Engine  Start 

Engine  Start  Dependency  Level 

Upgradeability 

Upgradeability  Level 

Styling 

Styling  Star  Value 

Deactivation  Capability 

Deactivation  Feature  Star  Value 

Endurance 

Minutes 

0.214C 

1 

1 

1 

1 

1 

1 

0,2140 

0,2140 

0,2140 

0,2140 

0,2140 

0,2140 

Thrust 

bs. 

Fuel  Efficiency 

Gallons  Per  F/H 

Speed 

Knots 

Range 

Miles 

Ceiling 

Feet 

Max  Take-off  Runway  Length 

Feet 

Power  Loading 

Power/Weight  Ratio 

Mean  Time  Between  Failures  (MTBF) 

Hours 

0.143C 

0,5352 

0,7135 

0,4037 

0,2043 

0,4636 

0,5677 

0,0765 

0,1020 

0,0577 

0,0292 

0,0663 

0,0812 

Mean  Time  Between  Maintenance  (MTBM) 

Hours 

0,1000 

0,3838 

0,5 

0,5 

0,7096 

0,5 

1 

0,0384 

0,0500 

0,0500 

0,0710 

0,0500 

0,1000 

Time  Between  Overhaul  (TBO) 

Hours 

0.043C 

1 

1 

1 

1 

1 

1 

0,0430 

0,0430 

0,0430 

0,0430 

0,0430 

0,0430 

Recording  Capacity 

Recording  Capacity  Level 

Maintenance  Specialty  Requirement 

Specialty  Certificate  Requirement  Level 

Civilian  Airports  Cross  Service 

Cross  Service  Support  Level 

Total  Safety  Incidents  (Class  A,B,C) 

Total  Number  Per  100K  F/H 

0,1420 

0 

0 

0 

0 

0 

0 

0,0000 

0,0000 

0,0000 

0,0000 

0,0000 

0,0000 

Engine 

Humber  of  Engines 

Ejection  Seat 

Ejection  Seat  Star  Value 

Hook 

Hook  Speed  Limit 

Drag  Chute 

Drag  Chute  Speed  limit 

Collision  Avoidance  System 

Avoidance  System  Star  value 

Recoverability 

Stall  Warning  Before  Stall  Margin  Knots 

Stability 

Degrees  of  Stability 

0,1430 

1 

1 

1 

1 

1 

1 

0,1430 

0,1430 

0,1430 

0,1430 

0,1430 

0,1430 

Maneuverability  /  Roll  Rate 

Degrees  Per  Second 

Maneuverability  /  G  Capacity 

3s 

Handling  Quality  Rating 

Cooper  Harper  Score 

Flight  Path  Stability 

Degrees  Per  Knot 

Instrument  Flight 

Instrument  Flight  Quality  Level 

Formation  Flight 

Formation  Flight  Quality  Level 

-ow  Level  Flight 

.ow  Level  Quality  Level 

Aerobatics 

Aerobatics  Quality  Level 

Ground  Handling 

Ground  Handling  Quality  Level 

Consistency  With  Current 

Currency  Scale 

HAVAIDS 

HAVAIDS  Scale 

Comm  System 

Comm  System  Quality  Level 

Radar  Capability  Level 

Maintenance  Cost 

Hundred  Dollars 

SUPPORTING  SYSTEMS 

Aircraft  Price 

Million  Dollars 

Synthetic  Training  System 

Synthetic  T rainer  Star  Value 

Debriefing  System 

Debriefing  System  Star  Value 

Life  Support  Materials  /  G-Suit 

G-Suit  Quality  Level 

KT-1C 

GLOBAL  WEIGHT  TOTAL - >  1,000  [RESULTING  SCORES . >|  I  0,6052 1  0,6991 1  0,701  1 1  III  I  II  "  | 


Table  3.6: 


DM-3  Alternatives  Scoring  Summary  List 


VALUE 


DESIGN  User  Friendly  /  Throttle 

User  Friendly  /  Avionics 
PERFORMANCE  Endurance 

RELIABILITY  Mean  Time  Between  Failures  (MTBF) 

MAINTAINABILITY  Mean  Time  Between  Maintenance  (MTBM) 
Time  Between  Overhaul  (TBO) 

SAFETY  Total  Safety  Incidents  (Class  A,B,C) 

FLYING  QUALITY  Stability 


MEASURE 


Throttle  Star  Vak 
Avionics  Scale 
Minutes 


0,1290 

0,2140 

0,1430 

0,1000 

0,0430 

0,1420 

0,1430 


ALTERNATIVE  SCORES 


T-34C  |  T-37B  |  T-6A  |  EMB.312  |  KT-1C  |  T-35 


0,7135  0,4037 


GLOBAL  WEIGHT  TOTAL  - 


WEIGHTED  SCORES 


T-34C  |  T-37B  |  T-6A 


0,0000 

0,0903 

0,2140 

0,0765 

0,0384 

0,0430 

0,0000 

0,1430 


0,0568 

0,0903 

0,2140 

0,1020 

0,0500 

0,0430 

0,0000 

0,1430 


0,0714 

0,1290 

0,2140 

0,0577 

0,0500 

0,0430 

0,0000 

0,1430 


0,0568 

0,1290 

0,2140 

0,0292 

0,0710 

0,0430 

0,0000 

0,1430 


KT-1C  |  T-35 


0,0860 

0,1290 

0,2140 

0,0663 

0,0500 

0,0430 

0,0000 

0,1430 


0,0568 

0,0903 

0,2140 

0,0812 

0,1000 

0,0430 

0,0000 

0,1430 


RESULTING  SCORES  - 


KT-1C 

|  0.60521  0,6991 1  0,7081 1  0,6859  gj), 731 3|  0.7282] 


After  constructing  all  the  individual  hierarchies  and  determining  the  evaluation 
measures,  the  group  came  together  on  the  13th  of  December,  2007  in  order  to  imple¬ 
ment  the  same  process  altogether.  After  creating  the  hierarchy,  the  group  was  asked 
to  weigh  the  value  increments  and  score  the  different  alternatives  in  the  same  way. 
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The  alternative  scorings  for  the  group  and  the  other  DMs  are  given  in  tables  in 
Appendix- A. 

3. 9  Summary 

This  chapter  covered  Steps  1  through  7  of  the  Value-Focused  Thinking  Process. 
It  presented  how  the  value  model  was  created  and  discussed  the  development  of  the 
evaluation  measures,  single  dimension  value  functions,  and  weighting  of  the  value 
hierarchy.  Chapter  4  will  discuss  steps  8  and  9  of  the  VFT  process  with  deterministic 
and  sensitivity  analysis  of  the  alternatives. 
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IV.  Results  and  Analysis 


4-1  Overview 

This  chapter  presents  the  analysis  of  the  model  that  was  described  in  Chapter 
3.  The  hierarchical  models  determine  a  rank  ordered  list  of  6  alternative  primary 
training  aircrafts  selected  for  the  study.  While  additional  alternatives  may  be  added 
at  any  time  in  order  to  view  the  ranking  amongst  the  previously  selected  alternatives, 
the  results  of  this  study  are  limited  to  the  real  alternatives  selected.  Overall  values  of 
each  alternative  for  both  the  individual  DMs  and  group  scoring  were  determined.  The 
alternatives  with  the  highest  values  are  the  most  preferred  primary  training  aircraft 
based  on  the  decision  makers  values.  A  sensitivity  analysis  was  also  performed  to  see 
how  sensitive  the  results  are  to  changes  in  weights  of  the  hierarchy. 

4-2  Step  8:  Deterministic  Analysis 

The  scores  from  the  measures  are  combined  to  form  an  overall  value  for  each 
of  the  alternatives.  Kirkwood  stated  that  overall  value  combined  from  the  measures 
represents  how  much  the  alternative  fulfills  the  objectives  of  the  decision  maker  [25]. 
The  overall  value  is  the  sum  of  the  values  of  each  measure  multiplied  by  the  global 
weight.  Below  is  group’s  overall  values  for  each  alternative  scoring  are  shown  as 
complete  list  in  Table  4.1  and  summary  list  in  Table  4.2. 


56 


Table  4.1:  Group  Alternatives  Scoring  Complete  List 


WEIGHTED  SCORES 

VALUE 

MEASURE 

Global  Weight 

T-34C 

IT-37B 

|T-6A 

IEMB.312  |KT-1C 

|T-35 

T-34C 

|T-37B 

|T-6A 

IEMB.312 

|KT-1C 

pM5 

DESIGN 

User  Friendly  /  Throttle 

Throttle  Star  Value 

0,0100 

0 

0,63 

0,75 

0,63 

1 

0,63 

0,0000 

0,0063 

0,0075 

0,0063 

0,0100 

0,0063 

User  Friendly  /  Avionics 

Avionics  Scale 

Simplicity  /  Standardized  Cockpit 

Standardization  Level 

0,0290 

0,8 

0,8 

1 

1 

1 

0,8 

0,0232 

0,0232 

0,0290 

0,0290 

0,0290 

0,0232 

Simplicity  /  Emergency  Procedures 

Robustness  /  Landing  Gears 

Average  Number  of  Steps  in  Boldfaces 

Failure  Rate  Per  100K  F/H 

Robustness  /  Airframe  Lifetime 

Years 

0,0580 

0,35 

1 

0,85 

0,6 

0,5 

0,5 

0,0203 

0,0580 

0,0493 

0,0348 

0,0290 

0,0290 

Ergonomics  /  Cockpit 

Cockpit  Star  Value 

0,0100 

0,8 

1 

0,8 

1 

1 

0,8 

0,0080 

0,0100 

0,0080 

0,0100 

0,0100 

0,0080 

Ergonomics  /  Visibility 

Visibility  Star  Value 

0,0190 

0,63 

0,63 

0,63 

0,63 

0,63 

0,63 

0,0120 

0,0120 

0,0120 

0,0120 

0,0120 

0,0120 

Ergonomics  /  Noise  Level 

Ergonomics  /  ECS 

dB 

ECS  Star  Value 

Systems  Complexity 

Systems  Dependency  /  Engine  Start 

Upgradeability 

Styling 

Deactivation  Capability 

Complexity  Level 

Engine  Start  Dependency  Level 

Upgradeability  Level 

Styling  Star  Value 

Deactivation  Feature  Star  Value 

PERFORMANCE 

Endurance 

Minutes 

Thrust 

Lbs. 

Fuel  Efficiency 

Gallons  Per  F/H 

0,1050 

1 

0 

0,084 

0 

0,145 

1 

0,1050 

0,0000 

0,0088 

0,0000 

0,0152 

0,1050 

Speed 

Range 

Ceiling 

Max  Take-off  Runway  Length 

Power  Loading 

Knots 

Feet 

Feet 

Power/Weight  Ratio 

RELIABILITY 

Mean  Time  Between  Failures  (MTBF) 

Hours 

MAINTAINABILITY 

Mean  Time  Between  Maintenance  (MTBM) 

Hours 

0,0570 

0,1169 

0,2781 

0,2781 

0,7883 

0,2781 

1 

0,0067 

0,0159 

0,0159 

0,0449 

0,0159 

0,0570 

Time  Between  Overhaul  (TBO) 

Hours 

0,0570 

0 

0 

0,75 

0 

0,5 

0 

0,0000 

0,0000 

0,0428 

0,0000 

0,0285 

0,0000 

Recording  Capacity 

Maintenance  Specialty  Requirement 

Recording  Capacity  Level 

Specialty  Certificate  Requirement  Level 

0,0110 

0,6 

0,6 

1 

0 

0 

0 

0,0066 

0,0066 

0,0110 

0,0000 

0,0000 

0,0000 

Civilian  Airports  Cross  Service 

Cross  Service  Support  Level 

SAFETY 

Total  Safety  Incidents  (Class  A,B,C) 

Total  Number  Per  100K  F/H 

Engine 

Number  of  Engines 

0,1320 

0 

0,8 

0 

0 

0 

0 

0,0000 

0,1056 

0,0000 

0,0000 

0,0000 

0,0000 

Ejection  Seat 

Ejection  Seat  Star  Value 

0,0660 

0 

0,71 

1 

0,71 

1 

0 

0,0000 

0,0469 

0,0660 

0,0469 

0,0660 

0,0000 

Hook 

Drag  Chute 

Hook  Speed  Limit 

Drag  Chute  Speed  limit 

Collision  Avoidance  System 

Avoidance  System  Star  value 

0,0130 

0,78 

0 

0,78 

0 

0,78 

0 

0,0101 

0,0000 

0,0101 

0,0000 

0,0101 

0,0000 

FLYING  QUALITY 

Recoverability 

Stall  Warning  Before  Stall  Margin_Knots 

0,1430 

0,89 

1 

0,89 

0,89 

0,89 

0,89 

0,1273 

0,1430 

0,1273 

0,1273 

0,1273 

0,1273 

Stability 

Maneuverability  /  Roll  Rate 

Maneuverability  /  G_Capacity 

Degrees  of  Stability 

Degrees  Per  Second 

Gs 

Handling  Quality  Rating 

Cooper  Harper  Score 

0,0360 

0,92 

0,97 

1 

0,97 

1 

0,97 

0,0331 

0,0349 

0,0360 

0,0349 

0,0360 

0,0349 

TRAINING  QUALITY 

Flight  Path  Stability 

Instrument  Flight 

Degrees  Per  Knot 

Instrument  Flight  Quality  Level 

Formation  Flight 

Formation  Flight  Quality  Level 

Low  Level  Flight 

Low  Level  Quality  Level 

Aerobatics 

Aerobatics  Quality  Level 

0,1580 

1 

1 

1 

1 

1 

1 

0,1580 

0,1580 

0,1580 

0,1580 

0,1580 

0,1580 

Ground  Handling 

Ground  Handling  Quality  Level 

TECHNOLOGY 

Consistency  With  Current 

Currency  Scale 

0,0320 

0,38 

0,63 

1 

1 

1 

0,38 

0,0122 

0,0202 

0,0320 

0,0320 

0,0320 

0,0122 

NAVAIDS 

NAVAIDS  Scale 

Comm  System 

Comm  System  Quality  Level 

Radar  Capability  Level 

COST 

Maintenance  Cost 

Hundred  Dollars 

0,0110 

0,664 

0,38 

0 

0,08 

0,021 

0,42 

0,0073 

0,0042 

0,0000 

0,0009 

0,0002 

0,0046 

Aircraft  Price 

Million  Dollars 

SUPPORTING  SYSTEMS 

Synthetic  Training  System 

Synthetic  Trainer  Star  Value 

0,0450 

0,33 

0,33 

1 

0 

1 

0 

0,0149 

0,0149 

0,0450 

0,0000 

0,0450 

0,0000 

Debriefing  System 

Debriefing  System  Star  Value 

0,0080 

0 

0 

0,74 

0 

0,74 

0 

0,0000 

0,0000 

0,0059 

0,0000 

0,0059 

0,0000 

Life  Support  Materials  /  G-Suit 

G-Suit  Quality  Level 

GLOBAL  WEIGHT  TOTAL- 


1, 000 


RESULTING  SCORES 


0,5446 1  0,6595 1  0,6645  0,5369|  0,6301 1  0,5774| 


Table  4.2:  Group  Alternatives  Scoring  Summary  List 


ALTERNATIVE  SCORES 

WEIGHTED  SCORES 

VALUE 

MEASURE 

Global  Weight 

T-34C 

IT-37B 

|T-6A 

IEMB.312 

|KT-1C 

|T-35 

T-34C 

|T-37B 

|T-6A 

IEMB.312 

|KT-1C 

|T-35 

DESIGN 

User  Friendly  /  Throttle 

Throttle  Star  Value 

0,0100 

0 

0,63 

0,75 

0,63 

1 

0,63 

0,0000 

0,0063 

0,0075 

0,0063 

0,0100 

0,0063 

Simplicity  /  Standardized  Cockpit 

Standardization  Level 

0,0290 

0,8 

0,8 

1 

1 

1 

0,8 

0,0232 

0,0232 

0,0290 

0,0290 

0,0290 

0,0232 

Robustness  /  Airframe  Lifetime 

Years 

0,0580 

0,35 

1 

0,85 

0,6 

0,5 

0,5 

0,0203 

0,0580 

0,0493 

0,0348 

0,0290 

0,0290 

Ergonomics  /  Cockpit 

Cockpit  Star  Value 

0,0100 

0,8 

1 

0,8 

1 

1 

0,8 

0,0080 

0,0100 

0,0080 

0,0100 

0,0100 

0,0080 

Ergonomics  /  Visibility 

Visibility  Star  Value 

0,0190 

0,63 

0,63 

0,63 

0,63 

0,63 

0,63 

0,0120 

0,0120 

0,0120 

0,0120 

0,0120 

0,0120 

Fuel  Efficiency 

Gallons  Per  F/H 

0,1050 

1 

0 

0,084 

0 

0,145 

1 

0,1050 

0,0000 

0,0088 

0,0000 

0,0152 

0,1050 

MAINTAINABILITY 

Mean  Time  Between  Maintenance  (MTBM) 

Hours 

0,0570 

0,1169 

0,2781 

0,2781 

0,7883 

0,2781 

1 

0,0067 

0,0159 

0,0159 

0,0449 

0,0159 

0,0570 

Time  Between  Overhaul  (TBO) 

Hours 

0,0570 

0 

0 

0,75 

0 

0,5 

0 

0,0000 

0,0000 

0,0428 

0,0000 

0,0285 

0,0000 

Maintenance  Specialty  Requirement 

Specialty  Certificate  Requirement  Level 

0,0110 

0,6 

0,6 

1 

0 

0 

0 

0,0066 

0,0066 

0,0110 

0,0000 

0,0000 

0,0000 

Engine 

Number  of  Engines 

0,1320 

0 

0,8 

0 

0 

0 

0 

0,0000 

0,1056 

0,0000 

0,0000 

0,0000 

0,0000 

Ejection  Seat 

Ejection  Seat  Star  Value 

0,0660 

0 

0,71 

1 

0,71 

1 

0 

0,0000 

0,0469 

0,0660 

0,0469 

0,0660 

0,0000 

Collision  Avoidance  System 

Avoidance  System  Star  value 

0,0130 

0,78 

0 

0,78 

0 

0,78 

0 

0,0101 

0,0000 

0,0101 

0,0000 

0,0101 

0,0000 

FLYING  QUALITY 

Recoverability 

Stall  Warning  Before  Stall  Margin  Knots 

0,1430 

0,89 

1 

0,89 

0,89 

0,89 

0,89 

0,1273 

0,1430 

0,1273 

0,1273 

0,1273 

0,1273 

Handling  Quality  Rating 

Cooper  Harper  Score 

0,0360 

0,92 

0,97 

1 

0,97 

1 

0,97 

0,0331 

0,0349 

0,0360 

0,0349 

0,0360 

0,0349 

Aerobatics 

Aerobatics  Quality  Level 

0,1580 

1 

1 

1 

1 

1 

1 

0,1580 

0,1580 

0,1580 

0,1580 

0,1580 

0,1580 

TECHNOLOGY 

Consistency  With  Current 

Currency  Scale 

0,0320 

0,38 

0,63 

1 

1 

1 

0,38 

0,0122 

0,0202 

0,0320 

0,0320 

0,0320 

0,0122 

COST 

Maintenance  Cost 

Hundred  Dollars 

0,0110 

0,664 

0,38 

0 

0,08 

0,021 

0,42 

0,0073 

0,0042 

0,0000 

0,0009 

0,0002 

0,0046 

SUPPORTING  SYSTEMS 

Synthetic  Training  System 

Synthetic  Trainer  Star  Value 

0,0450 

0,33 

0,33 

1 

0 

1 

0 

0,0149 

0,0149 

0,0450 

0,0000 

0,0450 

0,0000 

Debriefing  System 

Debriefing  System  Star  Value 

0,0080 

0 

0 

0,74 

0 

0,74 

0 

0,0000 

0,0000 

0,0059 

0,0000 

0,0059 

0,0000 

GLOBAL  WEIGHT  TOTAL - 


1,000 


RESULTING  SCORES 


0,5446  0,6595 


T-6A  TEXAN  I 

1 0,6645 


0,5369  0,6301  0,5774 


Table  4.1  shows  the  subtier  values  and  corresponding  measures  considered  by  the 
group  in  turquoise  color.  The  uncolored  lines  mean  that  they  weren’t  considered  by 
the  group  when  its  wish  list  was  created  in  order  to  bnild  the  group  hierarchy.  Right 
next  to  the  measures  are  the  global  weights  for  each  value  measure  to  be  multiplied 
by  each  alternative’s  corresponding  score.  Next  to  it,  the  weighted  scores  are  given. 
At  the  bottom  of  each  weighted  score  column  is  given  the  sum  of  the  weighted  value 
measure  scores  for  that  aircraft  type.  As  it’s  seen  in  the  purple-colored  cell,  T-6A 
Texan  II  is  the  alternative  with  the  highest  value  score  of  0.6645,  the  most  preferred 
primary  training  aircraft  for  the  group.  The  same  operation  has  been  done  for  all  the 
individual  DMs  and  the  results  in  summarized  form  have  been  shown  in  Appendix-A. 
All  the  scores  for  both  the  individual  DMs  and  group  have  been  gathered  below  in 
Table  4.3 


Table  4.3:  Group  &  Individual  DMs  Alternative  Scores 


ALTERNATIVES 

1 

T-34C 

2 

T-37B 

3 

T-6A 

4 

EMB.312 

5 

KT-1C 

6 

T-35 

DECISION  MAKERS 

GROUP 

0,5446 

0,6595 

0,6645 

0,5369 

0,6301 

0,5774 

DM-1 

0,2804 

0,3455 

0,3440 

0,2802 

0,3338 

0,3926 

DM-2 

0,3048 

0,7075 

0,5815 

0,5149 

0,5765 

0,3048 

DM-3 

0,6052 

0,6991 

0,7008 

0,6859 

0,7313 

0,7282 

DM-4 

0,3024 

0,4938 

0,5296 

0,4608 

0,5208 

0,3208 

DM-5 

0,2137 

0,3082 

0,3678 

0,3098 

0,3382 

0,3158 

DM-6 

0,7454 

0,7377 

0,8151 

0,7258 

0,7980 

0,7007 

DM-7 

0,5662 

0,7377 

0,8482 

0,6989 

0,8234 

0,4754 

DM-8 

0,4311 

0,7189 

0,7526 

0,5638 

0,6274 

0,3497 

DM-9 

0,5429 

0,5496 

0,6566 

0,5173 

0,7084 

0,4158 

DM-10 

0,4993 

0,6164 

0,6129 

0,5650 

0,5715 

0,5371 

DM-11 

0,7480 

0,6797 

0,9125 

0,9297 

0,9475 

0,7827 

DM-12 

0,5557 

0,4934 

0,4572 

0,2291 

0,5775 

0,4838 

DM-1 3 

0,5674 

0,5061 

0,7081 

0,5359 

0,7245 

0,5298 

DM-14 

0,5642 

0,6203 

0,8012 

0,5244 

0,7985 

0,4507 

DM-1 5 

0,3525 

0,4225 

0,3764 

0,3354 

0,3791 

0,2877 
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The  scores  in  Table  4.3  has  been  ranked  in  Table  4.4  from  best  to  worst,  the 
highest  score  (the  most  preferred)  being  1  and  the  lowest  score  (the  least  preferred) 
being  6.  For  the  analysis,  which  has  been  implemented  by  using  Kendall’s  Coefficient 
of  Concordance  in  this  study,  ranking  of  the  alternatives  is  the  only  information  to 
be  needed. 


Table  4.4:  Group  &  Individual  DMs  Alternative  Ranks 


ALTERNATIVES 

1 

2 

3 

4 

5 

6 

DECISION  MAKERS 

T-34C 

T-37B 

T-6A 

EMB.312 

KT-1C 

T-35 

GROUP 

5 

2 

1 

6 

3 

4 

DM-1 

5 

2 

3 

6 

4 

1 

DM-2 

5 

1 

2 

4 

3 

6 

DM-3 

6 

4 

3 

5 

1 

2 

DM-4 

6 

3 

1 

4 

2 

5 

DM-5 

6 

5 

1 

4 

2 

3 

DM-6 

3 

4 

1 

5 

2 

6 

DM-7 

5 

3 

1 

4 

2 

6 

DM-8 

5 

2 

1 

4 

3 

6 

DM-9 

4 

3 

2 

5 

1 

6 

DM-10 

6 

1 

2 

4 

3 

5 

DM-11 

5 

6 

3 

2 

1 

4 

DM-12 

2 

3 

5 

6 

1 

4 

DM-13 

3 

6 

2 

4 

1 

5 

DM-14 

4 

3 

1 

5 

2 

6 

DM-15 

4 

1 

3 

5 

2 

6 

Rank  Sum  (Rj) 

74 

49 

32 

73 

33 

75 

4-2.1  Nonparametric  Statistical  Methods.  The  ranking  of  the  alternatives 
was  analyzed  by  using  a  nonparametric  statistical  test,  Kendall’s  Coefficient  of  Con¬ 
cordance.  Siegel  defines  a  nonparametric  statistical  test  as:  ”a  test  whose  model  does 
not  specify  conditions  about  the  parameters  of  the  population  from  which  the  sample 
was  drawn”  [35].  Nonparametric  procedures  are  robust,  distribution- free  techniques 
that  are  particularly  useful  when  the  N  is  small,  as  in  this  case  where  there  are  only 
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6  subjects  (alternative  aircrafts).  There  are  many  additional  advantages  to  using 
nonparametric  procedures,  such  as  minimal  assumptions  concerning  the  underlying 
populations,  insensitivity  to  outliers,  and  the  ability  to  analyze  an  unequal  number 
of  judges  in  experimental  samples. 

Kendall’s  Coefficient  of  Concordance  (W)  was  used  to  analyze  these  results  to 
determine  if  the  DMs’  resulting  decisions  to  choose  the  best  alternative  are  similar 
to  each  other.  In  this  analysis,  n=6  and  m  varied  based  on  the  selected  sample  to 
be  tested.  All  15  individual  DMs  and  the  group  were  tested  against  one  another 
and  different  samples  were  tested  to  determine  if  there  is  any  consistency.  As  it  has 
been  stated  before,  the  main  purpose  of  the  research  was  to  determine  if  there  was 
any  concordance  among  the  decision  makers,  who  state  their  values  and  preferences 
individually,  and  the  group,  which  came  together  to  discuss  about  the  same  topic. 

These  experimental  samples  included  all  DMS  and  the  group,  USAF  pilots  and 
the  group,  USN  &  USMC  pilots  and  the  group,  TUAF  pilots  and  the  group,  instruc¬ 
tor  pilots  and  the  group,  non-instructor  pilots  and  the  group;  6  samples  in  total. 
Normally,  when  n<7  and  m<20,  the  ’’Table  of  Critical  Values  of  S  in  the  Kendall’s 
Coefficient  of  Concordance”  [35],  which  was  obtained  by  the  method  of  complete  per¬ 
mutations,  is  recommended  to  use  to  test  the  statistic  (Kendall’s  W)  for  statistical 
significance  according  to  Siegel  and  Castellan  [34],  However,  in  this  research,  both 
testing  the  significance  of  Kendall’s  coefficient  of  concordance  (W)  with  the  table  of 
critical  values  and  with  the  chi-square  approximation  test  of  the  sampling  distribution 
were  implemented  in  order  to  double-check  and  confirm  the  results. 
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Nonparametric  techniques  for  measuring  the  degree  of  correlation  method  are 
the  contingency  coefficient,  the  Crammer’  V  correlation  coefficient,  the  Spearman  rank 
correlation  coefficient,  the  Kendall  rank  correlation  coefficient,  the  Kendall  partial 
rank  correlation  coefficient,  Phi  correlation  coefficient  and  the  Kendall’s  coefficient  of 
concordance  (W). 

The  coefficient  of  contingency  and  the  Crammer’  V  correlation  coefficient  are 
uniquely  applicable  when  the  data  are  in  a  nominal  scale.  The  contingency  coefficient 
will  have  the  same  value  regardless  of  how  the  categories  are  arranged  in  the  rows 
and  columns. 

For  the  bivariate  case  two  rank  correlation  coefficient,  the  Spearman  rank  cor¬ 
relation  coefficient  and  the  Kendall  rank  correlation  coefficient  are  applicable.  The 
Spearman  rank  correlation  coefficient  is  somewhat  easier  to  compute.  However,  the 
Kendall  rank  correlation  coefficient  has  the  advantages  of  being  generalizable  to  a 
partial  correlation  coefficient.  Both  the  Spearman  rank  correlation  coefficient  and 
the  Kendall  rank  correlation  coefficient  have  a  sampling  distribution  which  is  prac¬ 
tically  indistinguishable  from  a  normal  distribution  for  large  sample  size  and  test  of 
the  significance  is  Z  -  test 

The  Kendall  partial  rank  correlation  coefficient  measures  the  degree  of  relation 
between  two  variables,  X  and  Y,  when  a  third  variable,  Z  is  held  constant.  This 
statistic  is  sometimes  called  Phi  correlation  coefficient. 
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The  Kendall’s  coefficient  of  concordance  (W)  measures  the  extent  of  agreement 
among  several  (m)  sets  of  ranking  of  n  entities.  Depending  on  the  application  field, 
the  entities  can  be  variables,  characters,  judges,  DMs  etc.  It  is  useful  in  determining 
the  agreement  among  several  judges  or  the  association  among  three  or  more  vari¬ 
ables.  Kendall’s  W  makes  no  assumptions  regarding  the  nature  of  the  probability 
distribution  and  can  handle  any  number  of  distinct  outcomes.  W  may  take  values 
only  between  0  and  +1.  This  is  because  when  more  than  two  judges  are  involved, 
agreement  and  disagreement  are  not  symmetrically  opposites. 

There  is  a  close  relationship  between  Friedman’s  two-way  analysis  of  variance 
without  replication  by  ranks  and  Kendall’s  coefficient  of  concordance.  They  address 
hypotheses  concerning  the  same  data  table  and  they  use  the  same  y2  statistic  for 
testing.  They  differ  only  in  the  formulation  of  their  respective  null  hypothesis.  In 
Friedman’s  test,  the  null  hypothesis  is  that,  n  objects  are  drawn  from  the  same  sta¬ 
tistical  population,  which  are  the  rows  of  the  data  table.  Under  H0,  they  should  have 
received  random  ranks  from  the  various  judges,  so  that  their  sums  of  ranks  should  be 
approximately  equal.  Kendall’s  test  focuses  on  the  m  judges.  If  the  null  hypothesis 
of  Friedman’s  test  is  true,  this  means  that  the  judges  have  produced  rankings  that 
are  unrelated  of  one  another.  This  is  the  null  hypothesis  of  Kendall’s  test  [34], 

*  Friedman’s  H$\  The  n  objects  are  drawn  from  the  same  statistical  population. 

*  Kendall’s  H0:  The  m  judges  produced  unrelated  rankings  of  the  objects. 
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Test  of  the  significance  of  Kendall’s  coefficient  of  concordance  is  Chi  -  square 
test  with  df  =  n-1. 

When  n<7  and  m<20  ,  Siegel  and  Castellan  [34]  recommended  using  their  table 
of  critical  values  for  S,  which  was  obtained  by  the  method  of  complete  permutations. 

Siegel  and  Castellan  listed  the  following  advantages  of  nonparametric  tests: 

1.  ”If  the  sample  size  is  very  small,  there  may  be  no  alternative  to  using  a  non¬ 
parametric  statistical  test  unless  the  nature  of  the  population  distribution  is 
known  exactly”. 

2.  ” Nonparametric  tests  typically  make  fewer  assumptions  about  the  data  and  may 
be  more  relevant  to  a  particular  situation.  In  addition,  the  hypothesis  tested  by 
the  nonparametric  test  may  be  more  appropriate  for  the  research  investigation” . 

3.  ’’Nonparametric  tests  are  available  to  analyze  data  which  are  inherently  in  ranks 
as  well  as  data  whose  seemingly  numerical  scores  have  the  strength  of  ranks. 
That  is,  the  researcher  may  only  be  able  to  say  of  his  or  her  subjects  that  one  has 
more  or  less  of  the  characteristic  than  another,  without  being  able  to  say  how 
much  more  or  less.  For  example,  in  studying  such  a  variable  as  anxiety,  we  may 
be  able  to  state  that  subject  A  is  more  anxious  than  subject  B  without  knowing 
at  all  exactly  how  much  more  anxious  A  is.  If  data  are  inherently  in  ranks,  or 
even  if  they  can  be  categorized  only  as  plus  or  minus  (more  or  less,  better  or 
worse),  they  can  be  treated  by  nonparametric  methods,  whereas  they  cannot 
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be  treated  by  parametric  methods  unless  precarious  and,  perhaps,  unrealistic 
assumptions  are  made  about  the  underlying  distributions” . 

4.  ”  Nonpar ametric  methods  are  available  to  treat  data  which  are  simply  categor¬ 
ical,  i.e.,  are  measured  in  a  nominal  scale.  No  parametric  technique  applies  to 
such  data”. 

5.  ”  There  are  suitable  nonparametric  statistical  tests  for  treating  samples  made 
up  of  observations  from  several  different  populations.  Parametric  tests  often 
cannot  handle  such  data  without  requiring  us  to  make  seemingly  unrealistic 
assumptions  or  requiring  cumbersome  computations” . 

6.  ” Nonparametric  statistical  tests  are  typically  much  easier  to  learn  and  to  apply 
than  are  parametric  tests.  In  addition,  their  interpretation  often  is  more  direct 
than  the  interpretation  of  parametric  tests” . 

Given  the  information  relating  to  nonparametric  statistical  methods,  the  focus 
returns  to  the  research  data.  In  this  study  the  degree  of  association  among  the 
rankings  of  alternative  aircrafts  of  the  18  DMs  was  measured.  According  to  Kendalls 
method,  the  alternatives  are  ranked  from  the  lowest  rank  sum  to  the  highest.  Thus 
T-6A  (with  a  rank  sum  of  32)  is  the  first,  followed  by  KT-1C,  then  T-37B,  then 
EMB.312,  then  T-34C,  and  the  last  alternative,  T-35  (with  a  rank  sum  of  75).  If  all 
16  DMs  each  ranked  the  six  aircrafts  in  the  same  order,  then  one  aircraft  would  have 
received  sixteen  ranks  of  1  and  thus  its  sum  of  ranks,  Rj,  would  be  l+l+...+l=16=m. 
The  least  promising  aircraft  would  have  i?j=6+6+...+6=96=6m.  In  fact,  with  perfect 
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agreement  among  the  executives,  the  various  sums  of  ranks,  Rj ,  would  be  these:  16, 
32,  48,  64,  80,  96,  though  not  necessarily  in  that  order.  In  general,  when  there  is  a 
perfect  agreement  among  k  sets  of  rankings,  we  get,  for  the  Rj,  the  series:  m,  2m,  3m, 
Nm.  On  the  other  hand,  if  there  had  been  no  agreement  among  the  three  executives, 
then  the  various  R/s  would  be  approximately  equal. 

The  following  steps  have  been  followed  to  compute  W: 

1)  Find  the  sum  of  ranks,  Rj,  in  (m  x  n  =  16  x  6)  table— >  74,  49,  32,  73,  33,  75. 

2)  Sum  the  Rj  and  divide  that  sum  by  ”n”  to  obtain  the  mean  value  of  the  Rj. 
— >  (74+49+32+73+33+75) =336  ,  (336/6)=  56.  (Another  way  to  compute  the  sum 
of  all  ranks  is  336=(16x6x7)/2  in  this  case,  or  in  general; 

m  x  n  x  (n  +  l)/2  (4.1) 

3)  Each  of  the  Rj  may  then  be  expressed  as  a  deviation  from  the  mean  value. 
The  deviations  of  the  rank  sums  from  this  mean  are:  18,  —7,  —24,  17,  —23,  and  19. 
(The  larger  are  these  deviations,  the  greater  is  the  degree  of  association  among  the  m 
sets  of  ranks.) 

3)  S,  the  sum  of  squares  of  these  deviations,  is  found.  S  represent  the  sum  of 
the  squares  of  the  deviations,  S=(324+49+576+289+529+361)=  2128  in  this  case. 
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4)  Knowing  the  observed  value  of  S,  the  value  of  W  for  the  data  can  be  found: 


W 


12  x  S 

m2  x  (n3  —  n) 


(4.2) 


12  x  2128 
162  x  (63  -  6) 


0.475 


(4.3) 


The  degree  of  agreement  among  the  16  DMs  is  reflected  by  the  degree  of  variance 
among  the  6  sums  of  ranks.  The  degree  of  agreement  among  the  16  DMs  in  ranking 
the  6  aircrafts  is  expressed  by  W=0.475.  The  Kendall  W  for  this  example  showed 
a  moderate  agreement  on  the  alternative  aircraft  rankings.  If  they  all  agree  W=l. 
According  to  Gibbons  and  Kendall,  ”if  they  differ  among  themselves  the  sums  of 
ranks  will  be  more  or  less  equal,  and  consequently  the  sum  of  squares  S  becomes 
small  compared  with  the  maximum  possible  value,  so  that  the  W  is  small.  As  W 
increases  from  0  to  1  the  deviations  become  more  different  and  there  is  a  greater 
measure  of  agreement  in  the  rankings.”  [19] 


4-2. 1.1  Testing  the  Significance  of  Kendall’s  Coefficient  of  Concordance 
(W)  with  the  Table  of  Critical  Values  for  S: 

The  significance  of  any  observed  value  of  W  can  be  tested  by  determining  the  proba¬ 
bility  associated  with  the  occurrence  under  Hq  of  a  value  as  large  as  the  S  with  which 
it  is  associated  [35]. 
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Ho=16  Sets  of  rankings  are  independent  by  taking  from  this  distribution  the 
probability  associated  wih  the  occurence  under  H0  of  a  value  as  large  as  an  observed 
sum  of  square  (S). 

By  this  method,  the  distribution  of  S  under  H0  has  been  worked  out  and  certain 
critical  values  have  been  tabled  by  Friedman  (1940),  which  then  extracted  by  Sidney 
Siegel  (” Nonparametric  Statistics  for  the  Behavioral  Sciences”,  Tablc-R:  Table  of 
Critical  Values  of  S  in  the  Kendall  Coefficient  of  Concordance).  The  table,  applicable 
for  m  from  3  to  20,  and  for  n  from  3  to  7,  gives  values  of  S  for  W’s  significance  at 
the  .05  and  .01  levels.  If  an  observed  S  is  equal  to  or  greater  than  that  shown  in  the 
table  for  a  particular  level  of  significance,  then  Hq  may  be  rejected  at  that  level  of 
significance. 

In  this  example  (all  DMs  and  the  group)  above  ,  m=16  DMs  ranked  n=6  air¬ 
crafts,  their  agreement  was  W=0.475  . 

H0:  W  =  0  ,  Hi.  W  ^  0 

•  The  Table  of  Critical  Values  of  S  [36]  in  Appendix- A  reveals  that  the  S  associated 
with  that  value  of  W  (S=2128)  is  significant  at  the  .05  level  since  it  is  greater 
than  Scrit  =  602.8  ; 

S  >  Scrit  =>•  REJECT  THE  NULL. 

•  S  associated  with  that  value  of  W  (S=2128)  is  significant  at  the  .01  level  since 
it  is  greater  than  Scrit  =  811 

S  >  Scrit  =>  REJECT  THE  NULL. 
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4-2. 1.2  Testing  the  Significance  of  Kendall’s  Coefficient  of  Concordance 
(W)  with  the  Chi-Square  Approximation  Test  of  the  Sampling  Distribution: 

When  exact  tables  for  W  (or  S)  are  not  available,  the  chi-square  distribution  provides 
a  reasonably  good  approximation  of  the  sampling  distribution  of  W.  The  chi-square 
approximation  of  the  sampling  distribution  of  W  is  computed  with  Equation  4.4.  The 
degrees  of  freedom  employed  for  Equation  4.4  are  dof=n-l. 

Xf  —  m  x  (n  —  1)  x  W  (4.4) 

If  the  value  of  \2F  as  computed  from  Equation  4.4  equals  or  exceeds  the  tabled 
value  from  the  Table  of  Critical  Values  of  Chi-Square  Distribution  (”  Nonparametric 
Statistics  for  the  Behavioral  Sciences”,  Page-249)  for  a  particular  level  of  significance 
and  a  particular  value  of  dof=n-l,  then  the  null  hypothesis,  H0=  m  rankings  are 
unrelated,  may  be  rejected  at  that  level  of  significance. 

In  our  sample  (all  DMs  and  the  group)  above  ,  m=16  DMs  ranked  n=6  aircrafts, 
their  agreement  was  W=0.475  . 

X%  =  16  x  (6  -  1)  x  0.475  =  38  (dof  =  6-1  =  5) 

The  value  Xf= 38  is  evaluated  with  the  Table  of  Critical  Values  of  Chi-Square 
Distribution  (” Nonparametric  Statistics  for  the  Behavioral  Sciences”,  Page-249). 

H0=  16  rankings  are  unrelated  (independent) 
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In  order  to  reject  the  null  hypothesis,  the  obtained  value  of  chi-square  must  be 
equal  to  or  greater  than  the  tabled  critical  value  at  the  prescribed  level  of  significance. 
For  dof=5,  the  tabled  critical  values  are  : 


Xom  =  11-07  (The  chi-square  value  at  the  95th  percentile) 

Xo.oi  =  15-09  (The  chi-square  value  at  the  99th  percentile) 

H0:  W  =  0  ,  Hr.  W  ^  0 

Since  Xf= 38  is  greater  than  both  of  the  aforementioned  critical  values  (y|-crit), 
the  null  hypothesis  H0=16  rankings  are  unrelated  (independent) is  REJECTED  at 
both  .05  and  .01  levels.  We  can  conclude  with  considerable  assurance  that  that  the 
agreement  among  the  16  DMs  is  higher  than  it  would  be  by  chance.  The  very  low 
probability  under  H0  associated  with  the  observed  value  of  W  enables  us  to  reject  the 
null  hypothesis  that  the  DMs’  ratings  are  unrelated  to  each  other. 

The  results  of  the  Kendall’s  Coefficient  of  Concordance  analysis  for  all  the  sam¬ 
ples  are  tabled  below  for  both  testing  the  significance  with  the  table  of  critical  values 
of  W  (Table  4.5)  and  with  the  chi-square  approximation  test  of  the  sampling  distri¬ 
bution  separately  (Table  4.6).  The  best  estimates  for  the  samples  according  to  the 
rankings  are  also  given  at  the  end  of  the  tables. 

As  it’s  seen  in  the  tables,  the  ’’best  estimate”  for  5  out  of  6  samples  (’all  DMs  + 
group’  sample,  ’USAF  DMs  +  group’  sample,  ’USN  &  USMC  DMs  +  group’  sample, 
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Table  4.5:  Testing  the  Significance  of  Kendall’s  Coefficient  of 
Concordance  with  the  Table  of  Critical  Values  for  S  -  Results 


CRITICAL  VALUE 
FOR  "S"  (Sent)  AT 
SIGNIFICANCE  LEVEL 

NULL  HYPOTHESIS 
(Ho)  FOR  "W"  AT 
SIGNIFICANCE  LEVEL 

BEST 

ALTERNATIVE 

AIRCRAFT 

NUMBER 

OF  DMs 

NUMBER 

OF  ALTs 

SAMPLE  NAME 

W 

S 

0.05 

0.01 

0.05 

0.01 

All  DMs+GROUP 

16 

6 

0.475 

2128 

602.8 

811.0 

REJECT 

REJECT 

USAF  DMs+GROUP 

6 

0.466071 

522 

299.0 

388.3 

REJECT 

REJECT 

USN&USMC  DMs+GROUP 

6 

0.634285 

277.5 

182.4 

229.4 

REJECT 

REJECT 

T-6A 

TUAF  DMs+GROUP 

6 

0.638857 

279.5 

182.4 

229.4 

REJECT 

REJECT 

KT-1C 

INSTRUCTOR  DMs+GROUP 

10 

6 

0.465143 

814 

376.7 

494.0 

REJECT 

REJECT 

T-6A 

NON-INSTRUCTOR  DMs+GROUP 

7 

6 

0.554519 

475.5 

260.2 

335.35 

REJECT 

REJECT 

T-6A 

Table  4.6:  Testing  the  Significance  of  Kendall’s  Coefficient  of 
Concordance  with  the  Chi-Square  Approximation  Test  of  the 
Sampling  Distribution  -  Results 


CRITICAL  VALUE 

FOR  X2  (X2=ri.)  AT 

SIGNIFICANCE  LEVEL 

NULL  HYPOTHESIS 
(HO)  FOR  "W"  AT 
SIGNIFICANCE  LEVEL 

BEST 

ALTERNATIVE 

AIRCRAFT 

NUMBER 

OF  DMs 

NUMBER 

OF  ALTs 

SAMPLE  NAME 

W 

■HEIBSlEa 

EEa 

0.05 

0.01 

0.05 

0.01 

UE1l.8(-fej| 

All  DMs+GROUP 

16 

6 

0.475 

5 

11.07 

15.09 

REJECT 

REJECT 

<  0.0001 

T-6A 

USAF  DMs+GROUP 

6 

0.466071 

18.64284 

5 

11.07 

15.09 

REJECT 

REJECT 

0.0023 

T-6A 

USN&USMC  DMs+GROUP 

6 

0.634285 

15.857125 

5 

11.07 

15.09 

REJECT 

REJECT 

0.0073 

T-6A 

TUAF  DMs+GROUP 

6 

0.638857 

15.971425 

5 

11.07 

15.09 

REJECT 

REJECT 

0.0069 

KT-1C 

INSTRUCTOR  DMs+GROUP 

10 

6 

0.465143 

23.25715 

5 

11.07 

15.09 

REJECT 

REJECT 

0.0003 

T-6A 

NON-INSTRUCTOR  DMs+GROUP 

7 

6 

0.554519 

19.408165 

5 

11.07 

15.09 

REJECT 

REJECT 

0.0016 

T-6A 

’instructor  DMs  +  group’  sample,  ’non-instructor  DMs  +  group’  sample)  is  T-6A, 
whereas  the  same  ’’best  estimate”  for  ’TUAF  DMs  +  group’  sample  is  KT-1C. 

Consequently,  it  can  be  concluded  at  the  end  of  both  significance  tests  that 
there  is  a  significant  association  among  the  16  DMs  (fifteen  individual  DMs  and  the 
group)  with  respect  to  how  they  rank  the  6  alternative  primary  training  aircrafts. 


4-2.2  Additional  Analysis.  After  accomplishing  the  analysis  for  testing  the 
significance  among  the  randomly  formed  samples,  each  including  ’’group”  in  it,  we 
decided  to  test  if  there  is  any  concordance  between  ’’the  group’s  ranking”  and  the 
resulting  ranking  of  the  other  DMs’  in  the  same  sample. 
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For  instance,  the  first  sample  was  ” all  DMs  +  group”  (including  16  rankings). 
We  wanted  to  see  if  there  is  any  concordance  between  group’s  ranking  and  the  total 
resulting  ranking  of  the  other  15  DM’s  ranking  together. 


Table  4.7:  Group  Ranking 


RANKING 

T-34C 

T-37B 

T-6A 

EMB.312 

KT-1C 

T-35 

GROUP 

5 

2 

1 

6 

3 

4 

The  rest  of  this  sample  was  all  of  the  15  individual  DMs.  Their  rankings  were 
combined  in  a  table  and  a  resulting  ranking  has  been  determined  based  on  their  sum 


of  ranks. 


Table  4.8:  All  DMs’  Resulting  Ranking 


RANKING 

T-34C 

T-37B 

T-6A 

EMB.312 

KT-1C 

T-35 

DM-1 

5 

2 

3 

6 

4 

1 

DM-2 

5 

1 

2 

4 

3 

6 

6 

4 

3 

5 

1 

2 

DM-4 

6 

3 

1 

4 

2 

5 

DM-5 

6 

5 

1 

4 

2 

3 

DM-6 

3 

4 

1 

5 

2 

6 

DM-7 

5 

3 

1 

4 

2 

6 

DM-8 

5 

2 

1 

4 

3 

6 

DM-9 

4 

3 

2 

5 

1 

6 

DM-10 

6 

1 

2 

4 

3 

5 

DM-1 1 

5 

6 

3 

2 

1 

4 

DM-12 

2 

3 

5 

6 

1 

4 

DM-1 3 

3 

6 

2 

4 

1 

5 

DM-14 

4 

3 

1 

5 

2 

6 

DM-1 5 

4 

1 

3 

5 

2 

6 

Total  of  Ranks 

69 

47 

31 

67 

30 

71 

RANKING 

T-34C 

T-37B 

T-6A 

EMB.312 

KT-1C 

T-35 

ALL  DMs 

5 

3 

2 

4 

1 

6 

Kendall’s  Coefficient  of  Concordance  (W)  had  been  used  for  the  previous  anal¬ 


ysis  since  there  were  more  than  two  rankings  in  all  of  those  6  samples.  But,  since 
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there  is  only  2  rankings  to  be  tested  here,  another  nonparametric  statistical  test, 
Kendall  Rank  Correlation  Coefficient  (r-tau),  has  been  used  to  test  whether  there  is 
any  relation  among  these  2  rankings. 

Kendall’s  tau  is  a  non-parametric  statistic  used  to  measure  the  degree  of  cor¬ 
respondence  (a  measure  of  correlation)  between  two  rankings  and  assessing  the  sig¬ 
nificance  of  this  correspondence  ,  and  so  measures  the  strength  of  the  relationship 
between  two  variables.  That  is,  for  each  variable  separately  the  values  are  put  in  or¬ 
der  and  numbered,  1  for  the  lowest  value,  2  for  the  next  lowest  and  so  on.  Siegel  [35] 
gives  details  of  how  to  calculate  Kendall’s  tau.  Kendall’s  tau  will  take  values  between 
+  1  and  -1,  with  a  positive  correlation  indicating  that  the  ranks  of  both  variables  in¬ 
crease  together  whilst  a  negative  correlation  indicates  that  as  the  rank  of  one  variable 
increases  the  other  one  decreases.  It  is  possible  to  calculate  confidence  intervals  and 
carry  out  hypothesis  tests  on  Kendall’s  tau.  The  main  advantages  of  using  Kendall’s 
tau  are  that  the  distribution  of  this  statistic  has  slightly  better  statistical  proper¬ 
ties  and  there  is  a  direct  interpretation  of  Kendall’s  tau  in  terms  of  probabilities  of 
observing  concordant  and  discordant  pairs. 

To  compute  tau,  the  order  of  the  ranks  should  be  rearranged,  so  that  the  rank¬ 
ings  on  the  primary  training  aircrafts  occur  in  the  natural  order: 


Table  4.9:  Group  versus  All  DMs  Ranking  -  Rearranged 


RANKING 

T-6A 

T-37B 

KT-1C 

T-35 

T-34C 

EMB.312 

GROUP 

1 

2 

3 

4 

5 

6 

ALL  DMs 

2 

3 

1 

6 

5 

4 
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Having  arranged  the  ranks  on  ”  Group  Ranking”  in  their  natural  order,  the  value 
of  S  is  determined  for  the  corresponding  order  of  ranks  on  ’’ALL  DMs  Ranking”. 

S  =  (4-1)  +  (3-1)  +  (3-0)  +  (0-2)  +  (0-1)  =  5 

(The  ’’All  DMs  Ranking”  which  is  farthest  to  the  left  is  2.  It  has  4  ranks  to  its 
right  which  are  larger  and  1  to  its  right  which  is  smaller,  so  that  its  contribution  to  S 
is  (4-1).  By  proceeding  in  this  way,  we  obtain  the  various  values  shown  above,  which 
we  have  summed  to  yield  S  =  5.)  Knowing  that  S  =  5  and  N  =  6  ,  we  may  use  the 
formula  below  to  compute  r  (tau): 


S 

(1/2)N(N  -  1) 


(4.5) 


5 

(l/2)6(6  —  1) 


0.33 


When  there  are  tied  (same  value)  observations,  then  t&  is  used: 


(4.6) 


n 


S 

J[N{N  -  l)/2  -  £*_1  U(U  -  l)/2]  [JV(JV  -  l)/2  —  £"=, -  l)/2] 


(4.7) 


where  R  is  the  number  of  observations  tied  at  a  particular  rank  and  u  is  the 
number  tied  at  a  rank. 
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Testing  the  Significance  of  Tau: 


Hq\  Two  sets  of  ranking  are  unrelated. 

H\ :  Two  sets  of  ranking  are  related  or  associated. 

When  N  is  10  or  less,  ”  Table  of  Probabilities  Associated  With  Values  As  Large 
As  Observed  Values  of  S  In  The  Kendall  Rank  Correlation  Coefficient”  (Nonpara- 
metric  Statistics  For  The  Behavioral  Sciences- Sidney  Siegel, 1956)  may  be  used  to 
determine  the  exact  probability  associated  with  the  occurrence  (one-tailed)  under  H0 
of  any  value  as  extreme  as  an  observed  S.  (The  sampling  distributions  of  S  and  r  are 
identical,  in  a  probability  sense.  Inasmuch  as  r  is  a  function  of  S,  either  might  be 
tabled.  It  is  more  convenient  to  tabulate  S.) 

For  such  small  samples,  the  significance  of  an  observed  relation  between  two 
samples  of  ranks  may  be  determined  by  simply  finding  the  value  of  S  and  then  referring 
to  the  table  to  determine  the  probability  (one-tailed)  associated  with  that  value.  If 
the  p  <  a,  H0  may  be  rejected. 

For  this  one,  S=5  and  N=  6  ;  table  shows  that  probability  of  occurrence  under 
H0  of  p  =  0.235 

Since  p=0.235  is  NOT  smaller  than  or  equal  to  a  =  0.05; 

we  ’’FAIL  TO  REJECT”  H0  at  this  level  of  significance. 

Since  p=0.235  is  NOT  smaller  than  or  equal  to  a  —  0.01; 

we  ’’FAIL  TO  REJECT”  Hq  at  this  level  of  significance. 
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The  other  rank  sets  to  be  tested  are  as  followed: 


Table  4.10:  Group  versus  USAF  DMs  Ranking 


RANKING 

T-6A 

T-37B 

KT-1C 

T-35 

T-34C 

EMB.312 

GROUP 

1 

2 

3 

4 

5 

6 

USAF  DMs 

1 

2 

1 

3 

5 

4 

Table  4.11:  Group  versus  USN  &  USMC  DMs  Ranking 


RANKING 

T-6A 

T-37B 

KT-1C 

T-35 

T-34C 

EMB.312 

GROUP 

1 

2 

3 

4 

5 

6 

USN&USMC  DMs 

1 

3 

2 

5 

4 

4 

Table  4.12:  Group  versus  TUAF  DMs  Ranking 


RANKING 

T-6A 

T-37B 

KT-1C 

T-35 

T-34C 

EMB.312 

GROUP 

1 

2 

3 

4 

5 

6 

TUAF  DMs 

2 

2 

1 

5 

3 

4 

Table  4.13:  Group  versus  Instructor  DMs  Ranking 


RANKING 

T-6A 

T-37B 

KT-1C 

T-35 

T-34C 

EMB.312 

GROUP 

1 

2 

3 

4 

5 

6 

INSTRUCTOR  DMs 

2 

3 

1 

4 

6 

5 

Table  4.14:  Group  versus  Non-Instructor  DMs  Ranking 


RANKING 

T-6A 

T-37B 

KT-1C 

T-35 

T-34C 

EMB.312 

GROUP 

1 

2 

3 

4 

5 

6 

NON-INSTRUCTOR  DMs 

1 

1 

1 

4 

2 

3 

The  results  of  all  of  those  significance  tests  are  shown  below  in  a  table: 


According  to  these  results,  we  don’t  have  enough  significance  to  reject  the  null 


hypothesis,  two  sets  of  rankings  are  unrelated  to  each  other.  However,  that  doesn’t 
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Table  4.15:  Kendall’s  Tau  Test  of  Significance  Results 


SETS  OF  RANKINGS 

TAU 

S 

P-Value 

a  =  0.05 

Group  vs  All  DMs 

0.33 

5 

0.235 

FAIL  TO  REJECT 

FAIL  TO  REJECT 

Group  vs  USAF  DMs 

0.69 

10 

0.056 

FAIL  TO  REJECT 

FAIL  TO  REJECT 

Group  vs  USN&USMC  DMs 

0.55 

8 

0.126 

FAIL  TO  REJECT 

FAIL  TO  REJECT 

Group  vs  TUAF  DMs 

0.41 

6 

0.251 

FAIL  TO  REJECT 

FAIL  TO  REJECT 

Group  vs  Instructor  DMs 

0.60 

9 

0.068 

FAIL  TO  REJECT 

FAIL  TO  REJECT 

Group  vs  Non-Instructor  DMs 

0.59 

8 

0.107 

FAIL  TO  REJECT 

FAIL  TO  REJECT 

necessarily  mean  that  the  null  hypothesis  is  true.  It  only  suggests  that  there  is  not 
sufficient  evidence  against  H0  in  favor  of  H1. 

There  is  one  important  fact  to  point  out  here  in  this  additional  study:  The 
sample  rankings  that  are  being  compared  to  group  ranking  are  a  resulting  ranking  of 
the  sum  of  the  mentioned  ranks.  And  these  samples  are  subsets  of  all  DMs  group.  It 
is  not  known  whether  summing  the  ranks  of  the  DMs’  in  the  sample  and  creating  a 
resulting  ranking  based  on  those  sums  causes  a  dependency  problem.  There  is  not  a 
clear  statement  relating  to  this  situation  in  the  literature.  Therefore,  more  research 
on  this  specific  matter  can  be  done  as  an  objective  of  a  future  study. 
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4-2.3  Conclusions  for  the  Deterministic  Analysis.  Siegel  cites  [35]  that 
Kendall  suggests,  ’’the  best  estimate  of  the  ’’true”  ranking  of  the  ”n”  objects  is  pro¬ 
vided,  when  W  is  significant,  by  the  order  of  the  various  sums  of  ranks,  Rf  [24],  If 
one  accepts  the  criterion  which  the  various  judges  have  agreed  upon  (as  evidenced  by 
the  magnitude  and  the  significance  of  W)  in  ranking  the  ”n”  entities,  the  best  esti¬ 
mate  of  the  ’’true”  ranking  of  those  entities  according  to  that  criterion  provided  by 
the  order  of  the  sums  of  ranks.  This  ’’best  estimate”  is  associated,  in  a  certain  sense, 
with  least  squares.  Thus  our  best  estimate  would  be  that  T-6A  should  be  chosen,  for 
Rj— 32,  the  lowest  value  observed,  for  the  ’’all  DMs  +  group”  sample. 

Given  the  sample  size,  Kendall’s  Coefficient  of  Condordance  (W)  was  used  to 
check  for  statistical  differences  between  the  individual  DMs’  rankings  and  group  rank¬ 
ing  as  a  nonparametric  statistical  method.  No  significant  differences  were  found  be¬ 
tween  the  individuals  and  group  (Kendall’s  coefficient  of  concordance,  alpha  =  0.05 
and  0.01). There  is  a  substantial  confidence  that  the  agreement  among  the  DMs  in 
these  samples  is  higher  than  it  would  be  by  chance.  All  of  the  probabilities  corre¬ 
sponding  to  the  samples  under  H0  (p-values)  associated  with  the  observed  value  of 
W  are  very  low  (<  0.01)which  enables  the  null  hypothesis, DMs’  ratings  are  unrelated 
to  each  other,  to  be  rejected. 

On  the  other  hand,  when  Kendall’s  Tan  statistic  was  used  (as  it’s  presented 
in  ’’Additional  Analysis”)  as  the  nonparametric  statistical  method  to  measure  the 
degree  of  correspondence  and  the  strength  of  the  relationship  between  two  rankings 
(i.e.  the  group  ranking  versus  USAF  DMs  sampling  group  ranking,  the  group  ranking 
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versus  instructor  DMs  sampling  group  ranking  etc.),  the  results  show  that  there  is  not 
sufficient  significance  to  reject  the  null  hypothesis:  two  sets  of  rankings  are  unrelated 
to  each  other.  However,  that  doesn’t  necessarily  mean  that  the  null  hypothesis  is 
true.  It  only  suggests  that  there  is  not  sufficient  evidence  against  Hq  in  favor  of 
H\ .  There  is  one  important  fact  to  point  out  here  in  this  additional  study.  The 
sample  group  rankings  that  are  being  compared  to  group  ranking  are  the  resulting 
rankings  of  the  sum  of  the  mentioned  ranks  belonging  to  the  individuals  who  formed 
the  sample  group.  And  these  sample  groups  are  a  subset  of  ’’all  DMs”  group.  It  is  not 
known  whether  summing  the  ranks  of  the  DMs’  in  those  sample  group  and  creating 
a  resulting  ranking  based  on  those  sums  causes  a  dependency  problem.  There  is  not 
a  clear  statement  relating  to  this  situation  in  the  literature.  Therefore,  more  research 
on  this  specific  matter  can  be  done  as  an  objective  of  a  future  study. 

When  we  examine  the  individual  DMs’  resulting  decision  for  the  best  primary 
training  aircraft  statistics,  T-6A  has  been  chosen  by  6  DMs,  KT-1C  has  been  chosen 
by  5  DMs,  T-37B  has  been  chosen  by  3  DMs  and  T-35  has  been  chosen  by  1  DM. 
T-35  is  in  a  position  of  outlier,  that  is  numerically  distant  from  the  rest  of  the  data. 
This  alternative  has  been  chosen  by  the  most  senior  officer  of  the  group.  As  it  has 
been  mentioned  in  the  earlier  discussion  of  group  decision  making  in  chapter-2;  the 
older,  more  experienced,  higher  ranked  are  decision  makers,  the  more  free  they  feel 
themselves  to  express  relatively  radical  or  extreme  opinions  in  comparison  to  the 
other  average  status  participants  of  the  decision  making  platform.  That  results  in 
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distinctive  decisions.  When  we  look  at  the  tables  again,  T-6A  has  come  up  as  the 
’’best  estimate”  for  5  samples  and  KT-1C  has  come  up  for  1  sample. 

The  group  decision  making  session  did  not  change  the  individual  DMs’  values 
and  preferences  toward  certain  alternative.  It  can  be  clearly  observed  that  the  tie 
between  the  two  leading  aircraft  remained  even  when  various  combination  of  decision 
maker  sample  groups  were  analyzed.  While  there  is  strong  concordance  among  the 
individual  DMs’  decisions  relating  to  the  same  topic,  the  group  decision  making  session 
took  three  times  longer  than  the  average  individual  DM  decision  process. 

An  interesting  observation  relating  to  the  group  decision  making  process  is  that, 
even  though  it  was  a  group  of  military  personnel  gathering  to  discuss  about  a  topic  and 
come  up  with  a  decision  and  the  senior  officers  are  supposed  to  lead  the  whole  session, 
it  hasn’t  been  this  way.  There  was  no  high-ranked  group  leader  pressure  during  the 
group  session  felt  by  the  members  of  the  group.  Because,  the  most  senior  officer  of 
the  group  did  not  express  any  kind  of  behavior  which  could  make  the  other  DMs  feel 
like  the  session  was  going  to  be  led  by  himself.  However  some  members  of  the  group 
did  not  fully  attend  the  process  other  than  observing  and  approving  the  ideas  with 
their  gestures  while  some  others  eagerly  tried  to  express  their  opinions  in  all  steps  of 
the  process.  After  creating  the  wish  list  at  the  beginning  of  the  session,  when  the 
discussion  relating  to  the  values  and  measures  started,  it  has  been  observed  that  some 
decision  makers  had  quite  a  big  disagreement  on  most  of  the  values.  So,  that  caused 
the  discussion  time  to  take  longer  than  it  was  planned.  Of  note,  instructor  pilots  in 
particular  expressed  very  similar  opinions  and  thoughts  throughout  the  session. 
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As  the  result  of  the  observations  and  analysis,  it  can  be  said  that  the  sam¬ 
ples’  decisions  highly  reflect  the  preferences  of  their  members.  Since,  there  is  such 
a  concordance  between  the  group  decision  and  decisions  of  the  individual  DMs  who 
constitute  the  group,  there  is  no  need  to  spend  more  time  to  come  up  with  almost 
the  similar  decision  in  a  group  environment  spending  a  huge  amount  of  valuable  time 
of  the  individuals  who  attend  the  group  meeting.  The  same  process  can  be  handled 
individually.  The  members  of  the  group  can  express  their  opinions  about  the  topic 
individually  and  the  process  can  be  handled  by  an  analyst  while  the  members  can 
keep  up  with  their  other  responsibilities.  That  would  be  a  very  time-efficient  decision 
process  at  the  end. 

The  results  show  that  there  is  a  high  consensus  in  the  samples  about  which 
aircraft  characteristics  are  important  for  the  preference  of  users  when  deciding  for  a 
primary  trainer.  Turkish  Air  Force  officers  were  the  only  group  which  came  up  with 
a  different  type  of  aircraft  as  a  primary  trainer  at  the  end  of  the  statistical  analysis. 
So,  it  can  be  concluded  that  the  samples’  decisions  will  reflect  different  requirements 
in  new  technology  (arm,  aircraft,  systems  etc.)  selection  resulting  from  the  country, 
region,  force  variety.  If  so,  the  individual  decision  makers  reflect  the  parallel  values 
and  preferences  of  the  group  to  which  they  belong.  That  means,  both  individual  or 
group  decisions  will  be  close  to  each  other. 

Consequently,  at  the  end  of  our  research  process;  the  group  seemed  to  produce 
parallel  decisions  with  the  individuals  who  constitute  it.  However,  groups  appear 
to  be  slower  in  reaching  decisions  than  individuals  are.  The  group  decision  can  be 
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produced  in  at  least  twice  the  amount  of  average  individual  decision  makers’s  decision 
process  time.  It  may  be  much  easier  and  less  wasteful  of  time  for  us  to  make  a  decision 
as  an  individual  than  to  involve  a  group.  Sometimes  individuals  working  alone  can 
be  more  effective  than  those  working  together  as  it  has  been  observed  during  our 
research.  There  were  some  individuals  as  it  has  been  expressed  before,  who  preferred 
not  to  contribute  or  present  an  opinion  at  all  during  the  group  meeting  even  though 
they  had  created  their  own  wish  lists  independently.  They  have  the  opinions  about 
the  issue,  yet  do  not  feel  themselves  either  comfortable  enough  or  a  real  part  of  the 
group  to  present  an  idea.  There  were  also  some  decision  makers  who  had  to  try  really 
hard  to  persuade  others  while  creating  the  value  hierarchy  and  evaluating  measures 
in  order  to  have  the  others  accepted  his  opinions  since  they  were  presenting  relatively 
different  opinions  then  the  majority  of  the  group.  And  some  of  those  quit  trying  hard 
after  some  point  and  did  not  continue  contributing  to  the  rest  of  the  process. 

The  quality  of  a  group  decision  varies  according  to  the  capabilities  of  its  members 
and  how  effectively  members  work  together.  It  is  a  very  important  factor  to  create 
a  worthy  result  at  the  end  of  the  group  meeting.  As  we  have  confirmed  during  onr 
observations,  we  can  not  expect  all  the  individuals  in  the  group  to  handle  the  issue 
during  the  meeting  as  well  as  they  do  it  individually.  Because,  they  are  taking  the 
whole  responsibility  of  the  result  while  they  are  handling  it  individually.  In  general, 
there  is  an  assumption  that  group  provides  more  important  resources  since  it  has 
greater  total  knowledge  and  information  than  its  individual  members.  But,  during 
onr  group  meeting  we  have  observed  the  opposite.  We  have  seen  that  the  group 
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resources  are  limited  to  its  members’  contribution  to  this.  If  a  member  does  not 


feel  comfortable  offering  new  ideas,  the  resources  will  be  missing  from  his  knowledge, 
values,  etc.  Further,  the  individual  behavior  of  members  can  affect  the  quality  of  the 
decision.  Some  tend  to  dominate  the  discussion,  preventing  others  from  participating. 

As  the  result  of  the  analysis  part,  we  can  state  with  strong  statistical  significance 
for  this  case  study  that  the  individual  DMs  can  create  very  close  decisions  to  each 
other  and  group.  The  resulting  suggestion  for  the  best  primary  training  aircraft  is 
T-6A  Texan  II. 

After  all  the  alternatives  were  generated  in  Step-6  and  scored  in  Step-7,  they 
were  measured  against  each  other.  Below,  Figure  4.1  displays  the  outputs  for  ’’Group” 
as  an  example  of  a  graph  of  comparing  alternatives.  The  larger  the  area  shown  in  a 
specific  color  correlates  to  the  larger  influence  that  the  value  has  on  specific  alternative. 

Examining  this  figure,  we  can  see  that  the  group  values  aerobatic  training  qual¬ 
ity  the  most  and  debriefing  system  the  least  in  a  primary  training  aircraft. 

4-3  Step  9:  Sensitivity  Analysis 

Sensitivity  analysis  can  be  viewed  in  Appendix-D. 
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Rankings  based  on  Top  Value 


T-6A 


T-37B 


KT-1C 


T-35 


T-34C 


EMB.312 


0.665 


0.657 


0.629 


0.578 


0.545 


0.536 


□  AEROBATICS  TRAINING  QUALITY  LEVEL 

□  STALL  WARNING  BEFORE  STALL  MARGIN  KNOTS 

■  NUMBER  OF  ENGINES 

■  GALLONS  PER  FLIGHT  HOUR 

■  EJECTION  SEAT  STAR  VALUE 

□  YEARS 

□  HOURS 
□ HOURSS 

□  SYNTHETIC  TRAINER  STAR  VALUE 

■  COOPER  HARPER  SCORE 
B  CURRENCY  SCALE 

□  STANDARDIZATION  LEVEL 

■  VISIBILITY  STAR  VALUE 

■  AVOIDANCE  SYSTEM  STAR  VALUE 

□  SPECIALTY  CERTIFICATE  REQUIREMENT  LEVEL 

■  HUNDRED  DOLLARS 

□  COCKPIT  STAR  VALUE 

■  THROTTLE  STAR  VALUE 

■  DEBRIEFING  SYSTEM  STAR  VALUE 


Figure  4.1:  Group  Rankings  based  on  Top  Value  (Best  Pri¬ 
mary  Training  Aircraft) 
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V.  Conclusions 


This  chapter  reviews  the  significance  of  this  research.  The  key  points  are  reviewed, 
the  significant  contributions  are  outlined,  and  recommendations  for  future  research 
are  suggested. 

5.1  Research 

The  research  in  this  thesis  explored  the  commonly  held  conception  that  decisions 
made  by  a  group  differ  from  the  decisions  made  by  the  individuals  who  form  this  group. 

First,  the  nature  and  the  scope  of  the  decision  making  problem  were  defined 
by  examining  the  individual  and  group  decisions,  advantages  and  disadvantages,  and 
the  comparison  of  the  natures  of  these  two  decision  processes.  A  group  decision  takes 
a  large  amount  of  time  and  effort,  it  requires  organization  and  consumes  valuable 
work  force  times  are  the  number  of  participants.  Second,  the  research  investigated 
the  concepts  of  the  value  focused  thinking  method  to  apply  the  procedure  with  a  case 
study  problem  in  order  to  assemble  the  required  data.  Lastly,  the  research  delved 
into  nonparametric  statistical  procedures  in  order  to  process,  analyze  and  compare 
the  data  coming  as  the  result  of  the  value  focused  thinking  method  in  the  case  study. 

The  value  focused  thinking  procedure  displays  similar  characteristics  to  any 
other  DA  applications. 

The  case  study  problem  was  quite  an  interesting  topic:  choosing  the  best  pri¬ 
mary  training  aircraft.  The  decision  of  choosing  a  primary  training  aircraft  to  train 
new  military  pilots  has  always  been  a  challenging  and  important  process  for  the  mil- 
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itary  forces  throughout  the  world.  What  are  the  characteristics  of  an  ideal  primary 
training  aircraft? 

Decision  makers  were  all  military  pilots  from  different  military  forces,  different 
career  fields  and  different  experience  levels.  They  have  either  been  in  such  a  procure¬ 
ment  process  earlier  or  can  expectedly  become  a  potential  member  of  any  procurement 
phase  for  the  forces  they  serve  sometime  in  their  future  career. 

The  primary  training  aircraft  decision  at  the  end  of  a  procurement  phase  influ¬ 
ences  a  great  number  of  pilot  generations  going  through  the  flight  training.  Therefore, 
this  decision  requires  attentive  contribution  of  each  steak  holder.  The  group  decision 
environment  may  not  always  be  the  ideal  atmosphere  to  encourage  the  members  to 
express  their  opinions  about  the  topic.  And  this  point  brings  us  to  the  main  purpose 
of  the  research:  Are  the  individual  decisions  really  that  much  different  from  the  group 
decision?  If  there  is  a  concordance  among  the  decisions  of  the  individuals  and  the 
group,  we  can  make  the  same  decisions  individually  and  the  analyst  can  implement 
this  process  for  all  the  stakeholders.  This  can  allow  the  people  to  fully  express  their 
real  values,  preferences  without  any  pressure  and  allow  them  to  accomplish  this  in  a 
much  shorter  time  in  comparison  to  the  group  decision  process. 

5.2  Contributions 

This  research  has  provided  some  important  contributions. 

The  first  contribution  is  that  it’s  an  experiment  based  on  a  real  world  problem 
analyzing  the  results  on  real  world  alternatives  at  the  end,  in  contrary  to  many  other 
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laboratory  experiments  based  either  on  a  statistical  urn  problem  or  a  monetary  policy 
experiment  in  the  literature  as  it  was  discussed  in  chapter-2. 

Second,  several  criteria  related  to  a  primary  training  aircraft  has  been  organized 
and  classified  in  this  study.  Additional  criteria  and  subcriteria  for  a  future  study  can 
be  added  at  any  time  and  the  margins  of  the  database  can  be  enlarged.  The  major 
guidelines  are  drawn  in  this  study. 

Third,  this  research  is  a  current  guide  on  the  primary  training  aircraft  selection 
covering  some  of  the  the  newest  types  as  the  alternatives  for  a  procurement  phase. 
The  database  relating  to  the  trainers  have  been  collected  through  January- February 
2008  from  several  sources. 

Fourth,  this  is  a  unique  study  observing  a  military  decision  making  process  in 
addition  to  statistically  testing  the  concordance  of  those  members  forming  the  group. 
Observing  the  group  atmosphere,  the  highest  ranked  member  of  the  group  meeting 
for  a  potential  leader  pressure  and  the  participation  of  the  individuals  in  the  group 
discussion  were  accomplished. 

The  research  method  incorporates  aspects  of  a  DA  method;  value  focused  think¬ 
ing,  and  a  nonparametric  statistical  methods.  The  VFT  process  is  performed  with 
any  individual  DM  and  gives  out  the  value  hierarchy  in  order  to  score  any  given  al¬ 
ternative.  New  DMs  be  added  for  a  future  research  at  any  point  and  the  size  of  the 
experiment  can  be  cnlaged. 
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Overall,  the  research  results  are  presented  to  recommend  a  primary  training 
aircraft  for  the  primary  flight  training.  This  study,  with  15  decision  makers,  can  give 
a  strong  insight  for  a  future  procurement  project  which  alternative  aircrafts  to  take 
into  account  for  the  evaluation.  In  addition  to  that,  since  the  results  of  the  study  are 
statistically  significant,  a  real  procurement  phase  for  any  type  of  the  aircraft  in  the 
future  can  be  implemented  with  an  extended  number  of  participants. 

5.3  Recommendations  for  Future  Work 

The  research  contained  within  this  thesis  may  be  extended  in  a  number  of 
directions.  Some  of  these  are: 

1.  The  experiment  can  be  improved  by  extending  the  number  of  DMs  participating 
in. 

2.  This  research  has  applied  the  method  in  the  order  of  interviewing  the  DMs 
first  and  then  implementing  the  group  meeting  and  observing  the  process.  The 
same  process  can  be  done  just  in  the  opposite  direction,  first  implementing  a 
group  meeting  and  then  interviewing  the  DMs  individually  afterward  to  get 
their  decisions.  And,  the  results  can  be  compared  to  the  results  of  this  study. 

3.  The  date  about  the  selected  alternatives  have  been  collected  from  several  sources 
(manufacturer  point  of  contacts,  Jane’s,  open  sources,  etc.).  It  would  be  more 
consistent  and  convenient  if  all  the  data  (at  least  the  data  for  a  single  value  for 
all  the  alternatives)  could  be  collected  from  the  same  source.  ( 


4.  The  DMs  in  the  study  had  widely  varied  backgrounds  (different  military  forces, 
ages,  experience  levels,  etc.), but  they  were  all  male  pilots  due  to  the  limited 
number  of  pilots  attending  AFIT  this  term,  where  the  DMs  of  this  study  have 
been  selected.  For  future  research,  it  would  be  desirable  to  include  female  pilots 
in  a  study,  thus;  demographic  variety  of  the  group  could  also  be  satisfied. 

5.4  Summary 

A  statistical  method  has  been  executed  in  order  to  explore  whether  there  is  any 
concordance  or  difference  between  the  individual  and  group  decisions  on  a  case  study: 
Best  primary  training  aircraft.  This  method  can  be  applied  to  the  other  aircraft  types 
as  well  as  other  case  study  fields.  The  conclusions  of  the  research  will  make  valuable 
contributions  to  the  previous  studies  and  findings  in  this  held. 

As  the  conclusion;  groups  produce  parallel  decisions  with  the  individuals  who 
constitute  them.  However,  groups  appear  to  be  slower  in  reaching  decisions  than 
individuals  are.  Since,  there  is  such  a  concordance  between  the  group  decision  and 
decisions  of  the  individual  DMs  who  constitute  the  group,  there  is  no  need  to  spend 
more  time  to  come  up  with  almost  the  similar  decision  in  a  group  environment  spend¬ 
ing  a  huge  amount  of  valuable  time  of  the  individuals  who  attend  the  group  meeting. 
The  members  of  the  group  can  express  their  opinions  about  the  topic  individually 
and  the  process  can  be  handled  by  an  analyst  while  the  members  can  keep  up  with 
their  other  responsibilities.  That  would  be  a  very  time-efficient  decision  process  at 
the  end. 
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Appendix  A.  Figures  &  Tables 


Tabic  A.l:  Subtier  Values- Evaluation  Measures  List 


SUBTIER  VALUE 

Measure 

SDVF 

User  Friendly  /  Throttle 

Throttle  Star  Value 

Categorical 

User  Friendly  /  Avionics 

Avionics  Scale 

Categorical 

Simplicity  /  Standardized  Cockpit 

Standardization  Level 

Categorical 

Simplicity  /  Emergency  Procedures 

Average  Number  of  Steps  in  Boldfaces 

Categorical 

Robustness  /  Landing  Gears 

Failure  Rate  Per  100K  F/H 

Categorical 

Robustness  /  Airframe  Lifetime 

Years 

Continuous 

Ergonomics  /  Cockpit 

Cockpit  Star  Value 

Categorical 

Ergonomics  /  Visibility 

Visibility  Star  Value 

Categorical 

Ergonomics  /  Noise  Level 

dB 

Continuous 

Ergonomics  /  ECS 

ECS  Star  Value 

Categorical 

Systems  Complexity 

Complexity  Level 

Categorical 

Systems  Dependency  /  Engine  Start 

Engine  Start  Dependency  Level 

Categorical 

Upgradeability 

Upgradeability  Level 

Categorical 

Styling 

Styling  Star  Value 

Categorical 

Deactivation  Capability 

Deactivation  Feature  Star  Value 

Categorical 

Endurance 

Minutes 

Continuous 

Thrust 

Lbs. 

Continuous 

Fuel  Efficiency 

Gallons  Per  F/H 

Continuous 

Speed 

Knots 

Continuous 

Range 

Miles 

Continuous 

Ceiling 

Feet 

Continuous 

Max  Take-off  Runway  Length 

Feet 

Continuous 

Power  Loading 

Power/Weight  Ratio 

Continuous 

Mean  Time  Between  Failures  (MTBF) 

Hours 

Continuous 

Mean  Time  Between  Maintenance  (MTBM) 

Hours 

Continuous 

Time  Between  Overhaul  (TBO) 

Hours 

Continuous 

Recording  Capacity 

Recording  Capacity  Level 

Categorical 

Maintenance  Specialty  Requirement 

Specialty  Certificate  Requirement  Level 

Categorical 

Civilian  Airports  Cross  Service 

Cross  Service  Support  Level 

Categorical 

Total  Safety  Incidents  (Class  A,B,C) 

Total  Number  Per  100K  F/H 

Categorical 

Engine 

Number  of  Engines 

Categorical 

Ejection  Seat 

Ejection  Seat  Star  Value 

Categorical 

Hook 

Hook  Speed  Limit 

Continuous 

Drag  Chute 

Drag  Chute  Speed  limit 

Continuous 

Collision  Avoidance  System 

Avoidance  System  Star  value 

Categorical 

Recoverability 

Stall  Warning  Before  Stall  MarginKnots 

Categorical 

Stability 

Degrees  of  Stability 

Categorical 

Maneuverability  /  Roll  Rate 

Degrees  Per  Second 

Continuous 

Maneuverability  /  G  Capacity 

Gs 

Categorical 

Handling  Quality  Rating 

Cooper  Harper  Score 

Categorical 

Flight  Path  Stability 

Degrees  Per  Knot 

Continuous 

Instrument  Flight 

Instrument  Flight  Quality  Level 

Categorical 

Formation  Flight 

Formation  Flight  Quality  Level 

Categorical 

Low  Level  Flight 

Low  Level  Quality  Level 

Categorical 

Aerobatics 

Aerobatics  Quality  Level 

Categorical 

Ground  Handling 

Ground  Handling  Quality  Level 

Categorical 

Consistency  With  Current 

Currency  Scale 

Categorical 

NAVAIDS 

NAVAIDS  Scale 

Categorical 

Comm  System 

Comm  System  Quality  Level 

Categorical 

Radar 

Radar  Capability  Level 

Categorical 

Maintenance  Cost 

Hundred  Dollars 

Continuous 

Aircraft  Price 

Million  Dollars 

Continuous 

Synthetic  Training  System 

Synthetic  Trainer  Star  Value 

Categorical 

Debriefing  System 

Debriefing  System  Star  Value 

Categorical 

Life  Support  Materials  /  G-Suit 

G-Suit  Quality  Level 

Categorical 
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Tabic  A. 2:  Performance-Evaluation  Measures  &  Scores  List 


Endurance 

Minutes 

#  minutes 

Thrust 

Lbs. 

#  lbs. 

Fuel  Efficiency 

Gallons  Per  F/H 

#  gallons  per  F/H 

Speed 

Knots 

#  Knots 

Range 

Miles 

#  NM 

Ceiling 

Feet 

#  Feet 

Max  Take-off  Runway  Length 

Feet 

#  Feet 

Power  Loading 

Power/Weight  Ratio 

#  (Ratio) 

Table  A. 3:  Reliability  &  Maintainability-Evaluation  Measures 

&  Scores  List 


Mean  Time  Between  Failures  (MTBF) 

Hours 

#  Hours 

Mean  Time  Between  Maintenance  (MTBM) 

Hours 

#  Hours 

Time  Between  Overhaul  (TBO) 

Hours 

#  Hours 

Recording  Capacity 

Recording  Capacity  Level 

1.  No  Recording  Capability 

2.  Cockpit  Voice  Recorder  (CVR) 

3.  Flight  Data  Recorder  (FDR) 

4.  CVR  +  FDR 

Maintenance  Specialty  Requirement 

Specialty  Certificate  Requirement  Level 

1 .  All  Specialty-Certificate  Maintenance  Requirement 

2.  No  Safety-Flight  Issues  Maintenance  Requirement 

3.  No  Specialty-Certificate  Maintenance  Requirement 

Civilian  Airports  Cross  Service 

Cross  Service  Support  Level 

1 .  No  Cross  Service 

2.  Stage-A  Cross  Servicing 

3.  Stage-B  Cross  Servicing 

Table  A. 4: 


Safety-Evaluation  Measures  &  Scores  List 


Total  Safety  Incidents  (Class  A,B,C) 

Total  Number  Per  100K  F/H 

#  of  incidents 

Engine 

Number  of  Engines 

#  of  engines 

Ejection  Seat 

Ejection  Seat  Star  Value 

1 .  No  Ejection  Seat 

2.  Ejection  Seat  With  a  Flight  Envelope 

3.  0-Feet  /  0-Knot 

Hook 

Hook  Speed  Limit 

#  Knots 

Drag  Chute 

Drag  Chute  Speed  limit 

#  Knots 

Collision  Avoidance  System 

Avoidance  System  Star  value 

1.  No  Avoidance  System 

2.  GCAS 

3.  TCAS 

4.  GCAS  +  TCAS 
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Table  A. 5:  Flying  Quality-Evaluation  Measures  &  Scores  List 


Recoverability 

Stall  Warning  Before  Stall  Margin_Knots 

1 .  0-Knot 

2.  5-Knots 

3.  10-Knots 

4.  15-Knots 

5.  20-Knots 

Stability 

Degrees  of  Stability 

1 .  Negatively  Stable 

2.  Neutrally  Stable 

3.  Normally  Stable 

4.  Totally  Stable 

Maneuverability  /  Roll  Rate 

Degrees  Per  Second 

#  of  degrees/second 

Maneuverability  /  G_Capacity 

Gs 

1.  +3/-2G 

2.  +5/-3G 

3.  +7/-5G 

4.  +9/-6G 

Handling  Quality  Rating 

Cooper  Harper  Score 

#  (1  -to-1 0) 

Flight  Path  Stability 

Degrees  Per  Knot 

#  of  degrees/knot 

Table  A. 6:  Training  Quality-Evaluation  Measures  &  Scores 

List 


Instrument  Flight 

Instrument  Flight  Quality  Level 

1 .  No  Instrument  Flight  Rating 

2.  Basic  Instrument  Flight  Training  Capability 

3.  Radio  Instrument  Flight  Training  Capability 

4.  No  Visual  /  IFR  Training  Capability 

Formation  Flight 

Formation  Flight  Quality  Level 

1 .  No  Formation  Flight 

2.  IFR  Formation  Flight  Training  Capability 

3.  Basic  Formation  Flight  Training  Capability 

4.  Cruise  Formation  Flight  Training  Capability 

Low  Level  Flight 

Low  Level  Quality  Level 

1 .  AGLIOOOFeet  300K 

2.  AGLIOOFeet  500K 

3.  AGL250Feet  500K 

4.  AGL250Feet  300K 

5.  AGL500Feet  300K 

6.  AGL500Feet  500K 

Aerobatics 

Aerobatics  Quality  Level 

1 .  No  Aerobatics  Capability 

2.  Chandelle 

3.  Chndll+  Hammerhead 

4.  Chndll+Hmmrhd+  Loop 

5.  Chndll+Hmmrhd+Loop+lmmelman 

6.  Chndll+Hmmrhd+Loop+lmmlmn+Cuban8 

7.  Chndll+Hmmrhd+Loop+lmmlmn+Cbn8+Spin 

Ground  Handling 

Ground  Handling  Quality  Level 

1 .  No  Steering  Capability 

2.  Reverse  Taxi  Capability 

3.  Steering  With  ON/OFF  Button 

4.  Steering  With  Press-Hold  Button 

5.  Castering  Nosewheel 

6.  Linked  Rudder  Pedals 
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Table  A.  7:  Technology  Quality- Evaluation  Measures  &  Scores 
List 


Consistency  With  Current 

Currency  Scale 

1.  Basic  Flight  Instruments 

2.  Basic  Fit  Inst  +  Basic  Nav  Sys 

3.  Basic  Fit  Inst  +  Adv  Nav  Sys 

4.  MFDs  +  Adv  Nav  Sys 

5.  Adv  Inst  Panel  +  Adv  Nav  Sys 

NAVAIDS 

NAVAIDS  Scale 

1.  No  Avionics 

2.  NDB 

3.  NDB+  DME 

4.  NDB+  TACAN 

5.  NDB+  TACAN  +  VOR/DME 

6.  TACAN  +  ILS 

7.  TACAN  +  ILS+  INS 

8.  GPS+  TACAN  +  ILS+  INS 

Comm  System 

Comm  System  Quality  Level 

1.  UHF  Only 

2.  VHF  Only 

3.  UHF  +  VHF 

4.  UHF  +  VHF  +  HaveQuick 

5.  UHF  +  VHF  +  Backup  Radio 

Radar 

Radar  Capability  Level 

1.  No  Radar  capability 

2.  Search-Only  Radar  Capability 

3.  Search&Track  Radar  Capability 

Table  A. 8:  Cost-Evaluation  Measures  &  Scores  List 


Maintenance  Cost 

Hundred  Dollars 

#  Hundred  dollars 

Aircraft  Price 

Million  Dollars 

#  Million  Dollars 

Table  A. 9: 
Scores  List 

Supporting  Systems- Evaluation  Measures  & 

Synthetic  Training  System 

Synthetic  T rainer  Star  Value 

1 .  No  Synthetic  T rainer 

2.  Part  Task  Trainer  (PTT) 

3.  Flight&Navigation  Procedures  Trainer  (FNPT) 

4.  Flight  Training  Device  (FTD) 

5.  Full  Simulator 

Debriefing  System 

Debriefing  System  Star  Value 

1 .  No  Debrief  System 

2.  GPS  Data  Recorder 

3.  Digital  Video  Debriefing  System  (DVDS) 

4.  ACMI  Debrifing  System 

5.  ACMI  +  DVDS 

Life  Support  Materials  /  G-Suit 

G-Suit  Quality  Level 

1.  No  Anti-G  Suit 

2.  Standart  Anti-G  Suit 

3.  Advanced  Technology  Anti-G  Suit 

93 


Table  A.  10: 


Data  Source  for  T-34C,  T-37B,  T-6A  (Pg-1/3) 


CD 

4* 


VALUE 

MEASURE 

T-34C 

T-37B 

T-6A 

DESIGN 

User  Friendly  /  Throttle 

Throttle  Star  Value 

Multilever  Engine  Control  (MLEC) 

SME  -  LCDR  John  ROTTER 

Mechanical  SLPC 

(Jane's  1977-1978  Pg-276) 

Digital  SLPC 

(Jane's  2007-2008  Pg-644) 

User  Friendly  /  Avionics 

Avionics  Scale 

Navigation  System  With  Auto  Alignment 

SME  -  LCDR  John  ROTTER 

Navigation  System  With  Auto  Alignment 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

Integrated  Avionics  System 
(Jane’s  2007-2008  Pg-645) 

Simplicity  /  Standardized  Cockpit 

Standardization  Level 

Integrated  Cockpit  (Pri  Inst  +  Essen  Nav  +  Essen  Comm) 

SME  -  LCDR  John  ROTTER 

Integrated  Cockpit  (Pri  Inst  +  Essen  Nav  +  Essen  Comm) 

(Jane’s  1977-1978  Pg-276) 

Advanced  Cockpit  Layout 
(Jane's  2007-2008  Pg-645) 

Simplicity  /  Emergency  Procedures 

Average  Number  of  Steps  in  Boldfaces 

6 

SME  -  LCDR  John  ROTTER 

3 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

2 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Robustness  /  Landing  Gears 

Failure  Rate  Per  100K  F/H 

5 

http://www.safetycenter.navy.mil/MEDIA/mech/issues/fall04/pdf/crossfe 

ed.pdf 

6 

http://findarticles.eom/p/articles/mi_m0IBT/is_6_57/ai_75645352 

2 

http://www.columbus.af.mil/shared/media/document/AFD-070501- 

074.pdf 

Robustness  /  Airframe  Lifetime 

v"m 

27  Years 

Tom  Calhoun 

Manager,  T-34/T-44/C-12  Programs 

Hawker  Beechcraft  Corporation 

40  Years 

T-37  AIRCRAFT  STRUCTURAL  INTEGRITY  PROGRAM 

MASTER  PLAN  UPDATE  -  FY  2003 

37  Years  (18,720  Flight  Hours) 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Ergonomics  /  Cockpit 

Cockpit  Star  Value 

Tandem 

(Jane's  1989-1990  Pg-349) 

Side-by-Side 

(Jane’s  1977-1978  Pg-276) 

Tandem 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Ergonomics  /  Visibility 

Visibility  Star  Value 

Clam-shell  Type  Canopy 

Tom  Calhoun 

Manager,  T-34/T-44/C-12  Programs 

Hawker  Beechcraft  Corporation 

Clam-shell  Type  Canopy 
(Jane’s  1977-1978  Pg-276) 

Clam-shell  Type  Canopy 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Ergonomics  /  Noise  Level 

dB 

105  dB 

The  Effect  of  Noise  Exposure  during  Primary  Flight  Training 

Ronald  M.  Robertson 

Naval  Aerospace  Medical  Research  Laboratory 

115  dB 

Defining  the  Cockpit  Noise  Hazard,  Aircrew  Hearing  Damage  Risk 

Miss  S.  James  Future  Systems  &  Technology  Division 

90  dB 

http://www.aviationweek.com/aw/qeneric/story  generic.jsp?channel=aw 
st&id=news/04283top.xml 

Ergonomics  /  ECS 

ECS  Star  Value 

50%  Ground  Efficient  +  In-Flight  Operated  ECS  System 

(Jane's  1989-1990  Pg-349) 

50%  Ground  Efficient*  In-Flight  Operated  ECS  System 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

Upgraded  ECS  to  Prevent  Ice  Accumulation, 

Vaporized  Air 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Systems  Complexity 

Complexity  Level 

Low  /  Moderate 

SME  -  LCDR  John  ROTTER 

Low  /  Moderate 

SME  -  1st  Lieutenant  Tufan  YELESER 

Moderate 

SME  -  LCDR  Theodore  DIAMOND 

Systems  Dependency  /  Engine  Start 

Engine  Start  Dependency  Level 

Battery-Power  Start 

SME  -  LCDR  John  ROTTER 

Battery  Power  Start 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

Battery  Power  Start 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Upgradeability 

Upgradeability  Level 

Modular  design  improved  upgradeability 

SME  -  LCDR  John  ROTTER 

Modular  design  improved  upgradeability 

SME  -  1st  Lieutenant  Tufan  YELESER 

Built-in  Upgradeability 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Styling 

Styling  Star  Value 

Round  Shape  Fuselage  with  Straight  Low  Wing 

SME  -  LCDR  John  ROTTER 

Round  Shape  Fuselage  with  Straight  Low  Wing 

SME  -  1st  Lieutenant  Tufan  YELESER 

Round  Shape  Fuselage  with  Straight  Low  Wing 

SME  -  LCDR  Theodore  DIAMOND 

Deactivation  Capability 

Deactivation  Feature  Star  Value 

NAVAIDS  (Flight  Instruments)  Deactivation  Capability 

SME  -  LCDR  John  ROTTER 

Intercom/Radio  System  Deactivation  Capability 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

MFD  Modes  Deactivation  Capability 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

PERFORMANCE 

Endurance 

Minutes 

285  min. 

Tom  Calhoun 

Manager,  T-34/T-44/C-12  Programs 

Hawker  Beechcraft  Corporation 

210  min. 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

180  min. 

(Jane's  2007-2008  Pg-645) 

Thrust 

1000  lbs. 

Tom  Calhoun 

Manager,  T-34/T-44/C-12  Programs 

Hawker  Beechcraft  Corporation 

2050  lbs. 

(Jane's  1977-1978  Pg-276) 

3400  lbs. 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Fuel  Efficiency 

Gallons  Per  F/H 

30  gallons/FH 

Tom  Calhoun 

Manager,  T-34/T-44/C-12  Programs 

Hawker  Beechcraft  Corporation 

83  gallons/FH  (600  Ibs/FH) 

T-37B  FLIGHT  MANUAL  T.O.  1T-37B-1 

55  gallons/FH  (400  Ibs/FH) 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Speed 

Knots 

182K 

(Jane's  1989-1990  Pg-349) 

349K 

(Jane's  1977-1978  Pg-276) 

350K 

(Jane's  2007-2008  Pg-645) 

Table  A.  11: 


Data  Source  for  T-34C 


,  T-37B,  T-6A  (Pg-2/3) 


Table  A.  12: 


Data  Source  for  T-34C,  T-37B,  T-6A  (Pg-3/3) 


Handling  Quality  Rating 

Cooper  Harper  Score 

SME  -  LCDR  John  ROTTER 

3 

SME  -  1st  Lieutenant  Tufan  YELESER 

2 

SME  -  LCDR  Theodore  DIAMOND 

Flight  Path  Stability 

Degrees  Per  Knot 

0.15  (Level-2) 

http://forumsjr-plane.org/lofiversion/index.php7f8.html 

0.15  (Level-2) 

http://www.pprune.org/forums/forumdisplay  .php?f=50 
http://forums.x-plane.org/lofiversion/index.php7f8.html 

0.06  (Level-1) 

http://www.pprune.org/forums/forumdisplay. php?f=50 

TRAINING  QUALITY 

Instrument  Flight 

Instrument  Flight  Quality  Level 

Radio  Instrument  Flight  Training 

SME  -  LCDR  John  ROTTER 

Radio  Instrument  Flight  Training 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

Radio  Instrument  Flight  Training 

SME  -  LCDR  Theodore  DIAMOND 

Formation  Flight 

Formation  Flight  Quality  Level 

Cruise  Formation  Flight  Training 

SME  -  LCDR  John  ROTTER 

Cruise  Formation  Flight  Training 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

Cruise  +  IFR  Formation  Flight  Training 

SME  -  LCDR  Theodore  DIAMOND 

Low  Level  Flight 

Low  Level  Quality  Level 

AGL-500'  /  300K 

SME  -  LCDR  John  ROTTER 

AGL-500'  /  300K 

SME  -  1st  Lieutenant  Tufan  YELESER 

AGL-500'  /  300K 

SME  -  LCDR  Theodore  DIAMOND 

Aerobatics 

Aerobatics  Quality  Level 

Chndll+Hmmrhd+Loop+lmmlmn+Cbn8+Spin 

SME  -  LCDR  John  ROTTER 

Chndll+Hmmrhd+Loop+lmmlmn+Cbn8+Spin 

SME  -  1st  Lieutenant  Tufan  YELESER 

Chndll+Hmmrhd+Loop+lmmlmn+Cbn8+Spin 

http://www.aviationweek.com/aw/generic/ 

story  generic.jsp?channel=awst&id=news/04283top.xml 

Ground  Handling 

Ground  Handling  Quality  Level 

Castering  Nosewheel  Steering  Capacity 

SME  -  LCDR  John  ROTTER 

Steering  w /  Press-Hold  Button 
(Jane's  1977-1978  Pg-276) 

Steering  With  ON/OFF  Button 

http://www.accessibleaviation.com/SUPT_Gouge/T6_Gouge 

TECHNOLOGY 

Consistency  With  Current 

Currency  Scale 

Basic  Flight  Instruments  +  Basic  Navigation  Systems 

SME  -  LCDR  John  ROTTER 

Basic  Flight  Instruments  +  Advanced  Navigation  Systems 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

Adv  Inst  Panel  +  Adv  Nav  Sys 

(Jane's  2007-2008  Pg-645) 

NAVAIDS 

NAVAIDS  Scale 

NDB+  TACAN+  VOR/DME 

TACAN  +  ILS 

(Jane's  1989-1990  Pg-349) 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

(Jane's  2007-2008  Pg-645) 

Comm  System 

Comm  System  Quality  Level 

VHF  only 

Tom  Calhoun 

Manager,  T-34/T-44/C-12  Programs 

Hawker  Beechcraft  Corporation 

UHF  only 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

UHF+VHF 

(Jane's  2007-2008  Pg-645) 

HaSa, 

Radar  Capability  Level 

No  Radar  Capability 

SME  -  LCDR  John  ROTTER 

No  Radar  Capability 

T-37B  FLIGHT  MANUAL  T.0. 1T-37B-1 

No  Radar  Capability 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

S  48  (0.5MMH/FH) 

$106  (1.10  MMH/FH) 

$  198  (2.06  MMH/FH) 

COST 

Maintenance  Cost 

Hundred  Dollars 

http://www.globalsecurity.org/military/library/policy/navy/ntsp/CXP-IFF- 

Improving  Aircraft  Accident  Forecasting 

Michael  T.  McNemey  and  Tracy  A.  Turen 

Center  for  Transportation  Research 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Aircraft  Price 

Million  Dollars 

S  1  M 

http://www.navy.mil/navydata/fact_display.asp?cid=1100&tid=1800&ct= 

1 

$  1 .24  M  (S180K  in  1960) 

Tony  Evans 

USAF  AETC  AETC/A4MAP 
http://www.dollartimes.com/calculators/inflation.htm 

The  Changing  Value  of  Dollar-lnflation  Calculator 

$  4.27  M 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

SUPPORTING  SYSTEMS 

Synthetic  Training  System 

Synthetic  Trainer  Star  Value 

Flight  &  Navigation  Procedures  Trainer 

SME  -  LCDR  John  ROTTER 

Flight  &  Navigation  Procedures  Trainer 

SME  -  1st  Lieutenant  Tufan  YELESER 

Full  Simulator 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

Debriefing  System 

Debriefing  System  Star  Value 

No  Debriefing  System 

SME  -  LCDR  John  ROTTER 

No  Debriefing  System 

SME  -  1st  Lieutenant  Tufan  YELESER 

Digital  Video  Debriefing  System  (DVDS) 
http://www.navair.navy.mil/publications/ar  1998.pdf 

Life  Support  Materials  /  G-Suit 

G-Suit  Quality  Level 

No  Anti-G  Suit 

SME  -  LCDR  John  ROTTER 

No  Anti-G  Suit 

SME  -  1st  Lieutenant  Tufan  YELESER 

Standart  Anti-G  Suit 

JD  O'Malley 

Regional  Manager,  North  America 

Hawker  Beechcraft  Corporation 

***  Pounds  are  converted  to  ga 

ons  with  the  following  conversion  ratio:  1  gallons  of  gas  =  3.785  pounds  of  gas 

"  Refrigerant  Type  (ECS):  The  mechanism  used  by  the  chiller,  which  performs  heat  transfer  by  converting  from  liquid  to  gas  and  gas  to  liquid  at  various  pressures  and  temperatures.  Common  refrigerants  used  in  commercial  HVAC  are  R-1 1 ,  R-12,  R-22,  R-1 13, 
"*  A  training  aircraft  is  assumed  to  fly  an  average  of  500  hours  a  year. 

**  According  to  Dr.  King's  conversion  formula,  the  thrust  measures  have  been  coverted  from  SHP  to  Lbs.  accord'ng  to  the  following  formula:  ]F(lbs)  *  V(ft/s)]  /  550  =  BHP 

**  According  to  http://forums.jetcareers.com/cfr-corner/43771-horsepower-to-thrust-conversion.html  F  (Lbs.)  =  SHP  *  375  *  Prop  efficiency  /  Speed  in  MPH 

’*  According  to  Cpt.  Jason  Brown's  thesis  database  relating  to  the  aircraft  maintenance  cost; 

---  E-5  labor  rate  per  hour  is  rounded  to  $  65  /  hour 

—  Depot  Maintenance  Activity  Group  (DMAG)  (aka... contractor)  is  $  127  /  hour 
**  According  to  those  two  value,  the  average  value  of  $  96  /  hour  has  been  used  in  this  study  for  Preventive  Maintenance  MMH/FH  cost. 


Table  A. 13:  Data  Source  for  EMB.312,  KT-1C,  T-35  (Pg-1/3) 


VALUE 

MEASURE 

EMB.312 

KT-1C 

T-35 

DESIGN 

User  Friendly  /  Throttle 

Throttle  Star  Value 

Mechanical  Single  SLPC 

Julian  Jaime  Cervantes 

Combined  (Mechanical+  Digital)  SLPC 

Mechanical  Single  SLPC 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-408) 

ENAER  Gerente  Comercial 

Integrated  Avionics  System 

Navigation  System  With  Auto  Alignment 

User  Friendly  /  Avionics 

Avionics  Scale 

Julian  Jaime  Cervantes 

Integrated  Avionics  System 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-407) 

ENAER  Gerente  Comercial 

Advanced  Cockpit  Layout 

Integrated  Cockpit  (Pri  Inst  +  Essen  Nav  +  Essen  Comm) 

Simplicity  /  Standardized  Cockpit 

Standardization  Level 

Julian  Jaime  Cervantes 

Advanced  Cockpit  Layout 

Ricardo  Klima  W. 

EMBRAER  Sales  Enaineerinq 

(Jane's  2007-2008  Pq-407) 

ENAER  Gerente  Comercial 

Simplicity  /  Emergency  Procedures 

Average  Number  of  Steps  in  Boldfaces 

6 

5 

6 

Julian  Jaime  Cervantes 

Jung  Seung-Lip 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

http://avitop.com/cs/forums/25/showforum.aspx 

ENAER  Gerente  Comercial 

Robustness  /  Landing  Gears 

Failure  Rate  Per  1 0OK  F/H 

5 

4 

3 

Julian  Jaime  Cervantes 

Jung  Seung-Lip 

Ricardo  Klima  W. 

http://avitop.com/cs/forums/25/showforum.aspx 

32  Years  (16000  Flight  Hours) 

30  Years  (15000  Flight  Hours) 

30  Years 

Robustness  /  Airframe  Lifetime 

Years 

Julian  Jaime  Cervantes 

Jung  Seung-Lip 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

http://avitop.com/cs/forums/25/showforum.aspx 

ENAER  Gerente  Comercial 

Stepped  Tandem 

Tandem 

Ergonomics  /  Cockpit 

Cockpit  Star  Value 

Julian  Jaime  Cervantes 

Stepped  Tandem 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-407) 

ENAER  Gerente  Comercial 

Ergonomics  /  Visibility 

Visibility  Star  Value 

Clam-shell  Type  Canopy 

Julian  Jaime  Cervantes 

Clam-Shell  Type  Canopy 

Clam-Shell  Type  Canopy 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-408) 

ENAER  Gerente  Comercial 

94  dB 

90  dB 

100  dB 

Ergonomics  /  Noise  Level 

dB 

Julian  Jaime  Cervantes 

A  noise  survey  conducted  on  the  KT-1C  at  KAI 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

M  Hancock,  A  Hazell,  M  Aitchison 

ENAER  Gerente  Comercial 

Full  Ground  +  In-Flight  ECS  System 

50%  Ground  Efficient+  In-Flight  Operated  ECS  System 

Ergonomics  /  ECS 

ECS  Star  Value 

Julian  Jaime  Cervantes 

Full  Ground  +  In-Flight  ECS  System 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-407) 

ENAER  Gerente  Comercial 

Moderate 

Moderate 

Low/Moderate 

Systems  Complexity 

Complexity  Level 

Julian  Jaime  Cervantes 

Kim  Chang-Sun 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

http://forum.dtmonline.com/leo/cgi-bin/printpage.cgi?forum=4&topic=475 

ENAER  Gerente  Comercial 

Battery  Power  Start 

Battery  Power  Start 

Battery  Power  Start 

Systems  Dependency  /  Engine  Start 

Engine  Start  Dependency  Level 

Julian  Jaime  Cervantes 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-408) 

ENAER  Gerente  Comercial 

Modular  design  improved  upgradeability 

Built-in  Upgradeability 

Modular  design  improved  upgradeability 

Upgradeability 

Upgradeability  Level 

Julian  Jaime  Cervantes 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

http://www.cmcelectronics.ca/ 

ENAER  Gerente  Comercial 

Round  Shape  Fuselage  with  Straight  Low  Wine 

Round  Shape  Fuselage  with  Straight  Low  Wing 

Round  Shape  Fuselage  with  Straight  Low  Wing 

Styling 

Styling  Star  Value 

Julian  Jaime  Cervantes 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

http://www.flugrevue.de/index.php?id=228&f=1283&a=td 

ENAER  Gerente  Comercial 

NAVAIDS  (Flight  Instruments)  Deactivation  Capability 

Deactivation  Capability 

Deactivation  Feature  Star  Value 

NAVAIDS  (Flight  Instruments)  Deactivation  Capability 

Julian  Jaime  Cervantes 

MFD  Modes  Deactivation  Capability 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

http://www.cmcelectronics.ca/ 

ENAER  Gerente  Comercial 

300  min. 

340  min. 

PERFORMANCE 

Endurance 

Minutes 

Julian  Jaime  Cervantes 

360  min. 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-409) 

ENAER  Gerente  Comercial 

970  lbs.  (750  shp) 

1017  lbs.  (950  shp) 

543  lbs.  (300  bhp) 

Thrust 

Lbs. 

Julian  Jaime  Cervantes 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-408) 

ENAER  Gerente  Comercial 

22  gallons/FH  (85  It/FH) 

Fuel  Efficiency 

Gallons  Per  F/H 

61  gallons/FH  (445  Ibs/FH)  (-0.595  Ib/SHP/h) 

Julian  Jaime  Cervantes 

52  gallons/FH  (380  Ibs/FH) 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

http://www.armada.ch/06-4/article-full.cfm 

ENAER  Gerente  Comercial 

290  K 

180K 

Speed 

Knots 

Julian  Jaime  Cervantes 

350K 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-408) 

ENAER  Gerente  Comercial 

995  NM 

680  NM 

Range 

Miles 

Julian  Jaime  Cervantes 

720  NM 

Ricardo  Klima  W. 

EMBRAER  Sales  Enqineerinq 

(Jane's  2007-2008  Pq-408) 

ENAER  Gerente  Comercial 

25050  ft 

19160 

Ceiling 

Feet 

Julian  Jaime  Cervantes 

38000  ft 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-408) 

ENAER  Gerente  Comercial 

2086  ft  (636m) 

940  ft 

Max  T ake-off  Runway  Length 

Feet 

Julian  Jaime  Cervantes 

1620  ft. 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pg-409) 

ENAER  Gerente  Comercial 

0.243  (970  lbs  /  3990  pounds)  0.294  SHP/kg 

0.261  (543  lbs  /  2080  pounds)  300  bhp  /  2950  lb 

Power  Loading 

Power/Weight  Ratio 

Julian  Jaime  Cervantes 

0.241  (1017  lbs  /  4210pounds)  3.59  kg/kW  (5.89  Ib/shp) 

Ricardo  Klima  W. 

EMBRAER  Sales  Engineering 

[Jane's  2007-2008  Pg-408) 

ENAER  Gerente  Comercial 

Table  A. 14:  Data  Source  for  EMB.312,  KT-1C,  T-35  (Pg-2/3) 


34  FH 

RELIABILITY 

Mean  Time  Between  Failures  (MTBF) 

Hours 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Choi  Jung-Keun 

http://jets.dk/cs/forums/25/ShowForum.aspx 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

MAINTAINABILITY 

Mean  Time  Between  Maintenance  (MTBM) 

Hour. 

13.9  FH 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

10  FH 

Choi  Jung-Keun 

http://jets.dk/cs/forums/25/ShowForum.aspx 

100  FH 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

3000  Flight  Hours  TBO  for  Engine 

4000  Flight  Hours  TBO  for  Engine 

1400  Flight  Hours  TBO  for  Engine 

Time  Between  Overhaul  (TBO) 

Hours 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Choi  Jung-Keun 

http://jets.dk/cs/forums/25/ShowForum.aspx 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Recording  Capacity 

Recording  Capacity  Level 

No  Recording  Capability 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

FDR 

(Jane's  2007-2008  Pg-408) 

No  Recording  Capability 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Maintenance  Specialty  Requirement 

Specialty  Certificate  Requirement  Level 

All  Specialty-Certificate  Maintenance  Requirement 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineerinq 

All  Specialty-Certificate  Maintenance  Requirement 

http://www.fluqrevue.de/index.php?id=228&f=1283&a=td 

All  Specialty-Certificate  Maintenance  Requirement 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Civilian  Airports  Cross  Service 

Cross  Service  Support  Level 

No  Cross-Serviceable 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Enqineerinq 

No  Cross-Serviceable 

http://www.fluqrevue.de/index.php?id=228&f=1283&a=td 

No  Cross-Serviceable 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

SAFETY 

Total  Safety  Incidents  (Class  A,B.C) 

Total  Number  Per  100K  F/H 

41 

http://www.whq- 

forum.de/invisionboard/lofiversion/index.php/tl  9777-1450.html 

14 

Kim  Chang-Sun 

http://forum.dtmonline.com/leo/cgi-bin/printpage.cgi?forum=4&topic=475 

36 

Ricardo  Klima  W. 

Engine 

Number  of  Engines 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

(Jane's  2007-2008  Pq-408) 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Ejection  Seat 

Ejection  Seat  Star  Value 

Ejection  Seat  with  a  Flight  Envelope  (MBBR8LC) 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Zero/Zero  Ejection  Seat 
(Jane's  2007-2008  Pg-408) 

No  Ejection  Seat 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Hook 

Hook  Speed  Limit 

N/A 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

N/A 

(Jane's  2007-2008  Pg-408) 

N/A 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Drag  Chute 

Drag  Chute  Speed  limit 

N/A 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

N/A 

(Jane's  2007-2008  Pg-408) 

N/A 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Collision  Avoidance  System 

Avoidance  System  Star  value 

No  Avoidance  System 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

TCAS 

http://acilr-cdril.com/CD  Nol/CMC  capability  brochure  en.pdf 

No  Avoidance  System 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

FLYING  QUALITY 

Recoverability 

Stall  Warning  Before  Stall  Margin_Knots 

5  K 

Julian  Jaime  Cervantes 

5  K 

Kwang  Weon-Kee 

5  K 

Ricardo  Klima  W. 

Stability 

Degrees  of  Stability 

Normally  (positively)  stable 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Normally  (positively)  stable 

http://www.fluqrevue.de/index.php?id=228&f=1283&a=td 

Normally  (positively)  stable 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Maneuverability  /  Roll  Rate 

Degrees  Per  Second 

230  degrees/sec 

http://www.milavia.net/news/archive.php72005-12 

250  degrees/sec 

Kwang  Weon-Kee 

http://avitop.com/cs/forums/thread/5195.aspx 

111  degrees/sec 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Maneuverability  /  G_Capacity 

Gs 

+  6  /  -3 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

+7  /  -3.5 

(Jane's  2007-2008  Pg-409) 

+6/ -3 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Handling  Quality  Rating 

Cooper  Harper  Score 

3 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

2 

Moon  Bong-Choi 

http://www.usaviation.com/forums/index.php?showtopic=40069&pid=572116&start=0&# 

entry572116 

3 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Flight  Path  Stability 

Degrees  Per  Knot 

0.06  (Level-1) 

http://www.pprune.org/forums/forumdisplay.php?f=50 

0.06  (Level-1) 

http://www.pprune.org/forums/forumdisplay.php?f=50 

0.15  (Level-2) 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

TRAINING  QUALITY 

Instrument  Flight 

Instrument  Flight  Quality  Level 

Radio  Instrument  Flight  Training 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Radio  Instrument  Flight  Training 

Jeon  Eunkyhung 

http://boardreader.com/fp/US_Aviation_Aviation_Forums_Ai_21840/Flight_Training_For 
urn  186922.html 

Radio  Instrument  Flight  Training 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Formation  Flight 

Formation  Flight  Quality  Level 

Cruise  Formation  Flight  Training 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Cruise  +  IFR  Formation  Flight  Training 

Jeon  Eunkyhung 

http://boardreader.com/fp/US_Aviation_Aviation_Forums_Ai_21840/Flight_Training_For 
urn  186922.html 

Cruise  Formation  Flight  Training 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Low  Level  Flight 

Low  Level  Quality  Level 

AGL-500'  /  300K 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

AGL-500'  /  300K 

Jeon  Eunkyhung 

http://boardreader.com/fp/US_Aviation_Aviation_Forums_Ai_21840/Flight_Training_For 
urn  186922.html 

AGL-500'  /  300K 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Aerobatics 

Aerobatics  Quality  Level 

Chndll+Hmmrhd+Loop+lmmlmn+Cbn8+Spin 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Chndll+Hmmrhd+Loop+lmmlmn+Cbn8+Spin 
(Jane's  2007-2008  Pg-408) 

Chndll+Hmmrhd+Loop+lmmlmn+Cbn8+Spin 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Ground  Handling 

Ground  Handling  Quality  Level 

Steering  With  ON/OFF  Button 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Steering  With  ON/OFF  Button 
(Jane's  2007-2008  Pq-408) 

Differential  Braking 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

TECHNOLOGY 

Consistency  With  Current 

Currency  Scale 

Adv  Inst  Panel  +  Adv  Nav  Sys 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Adv  Inst  Panel  +  Adv  Nav  Sys 
(Jane's  2007-2008  Pg-408) 

Basic  Flight  Instruments  +  Basic  Navigation  Systems 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Table  A.  15 


Data  Source  for  EMB.312,  KT-1C,  T-35  (Pg-3/3) 


CD 

CD 


NAVAIDS 

NAVAIDS  Scale 

TACAN+ILS  (ILS,VOR,DME) 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

GPS+  TACAN+  ILS+  INS 
(Jane's  2007-2008  Pg-407) 

NDB  +  VOR/DME 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Comm  System 

Comm  System  Quality  Level 

UHF+ VHF 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

VHF  only 

(Jane's  2007-2008  Pg-408) 

VHF  only 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Radar 

Radar  Capability  Level 

No  Radar  Capability 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

No  Radar  capability 

http://www.thefreelibrary.com/A+clash+of+generations:+the+key+question+in+primary+tr 

ainers 

No  Radar  Capability 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

$120  (1.25  MH/FH) 

$  140  (1.46  MMH/FH) 

$67  (0.7  MMH/FH) 

COST 

Maintenance  Cost 

Hundred  Dollars 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Jeung  Eui-Tae 

http://aviationforum.org/fomms/showth  read.php?threadid=3299 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Aircraft  Price 

Million  Dollars 

$2  M 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

$  5M 

(Jane's  2007-2008  Pq-407) 

$  685K 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

SUPPORTING  SYSTEMS 

Synthetic  Training  System 

Synthetic  Trainer  Star  Value 

No  Synthetic  Trainer  system 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Full  Simulator 

http://www.flugrevue.de/index.php?id=228&f=1283&a=td 

No  Synthetic  T rainer  System 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Debriefing  System 

Debriefing  System  Star  Value 

No  Debriefing  System 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Digital  Video  Debriefing  System  (DVDS) 
http://www.flugrevue.de/index.php?id=228&f=1283&a=td 

No  Debrifing  System 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

Life  Support  Materials  /  G-Suit 

G-Suit  Quality  Level 

No  Anti-G  Suit 

Julian  Jaime  Cervantes 

EMBRAER  Sales  Engineering 

Standard  Anti-G  Suit  System 

http://www.aving.net/kr/news/default.asp?mode=read&c_num=62549&C_Code=07&SP_ 

Num=116 

No  Anti-G  Suit  System 

Ricardo  Klima  W. 

ENAER  Gerente  Comercial 

***  Pounds  are  converted  to  gallons  with  the  following  conversion  ratio:  1  gallons  of  gas  =  3.785  pounds  of  gas 

***  Refrigerant  Type  (ECS):  The  mechanism  used  by  the  chiller,  which  performs  heat  transfer  by  converting  from  liquid  to  gas  and  gas  to  liquid  at  various  pressures  and  temperatures.  Common  refrigerants  used  in  commercial  HVAC  are  R-1 1 ,  R-12,  R-22,  R-1 13, 
***  A  training  aircraft  is  assumed  to  fly  an  average  of  500  hours  a  year. 

***  According  to  Dr.  King's  conversion  formula,  the  thrust  measures  have  been  coverted  from  SHP  to  Lbs.  accord'ng  to  the  following  formula:  ]F(lbs)  *  V(ft/s)J  /  550  =  BHP 

***  According  to  http://forums.jetcareers.com/cfi-corner/43771-horsepower-to-thmst-conversion.html  F  (Lbs.)  =  SHP  *  375  *  Prop  efficiency  /  Speed  in  MPH 

***  According  to  Cpt.  Jason  Brown's  thesis  database  relating  to  the  aircraft  maintenance  cost; 

—  E-5  labor  rate  per  hour  is  rounded  to  $  65  /  hour 

—  Depot  Maintenance  Activity  Group  (DMAG)  (aka... contractor)  is  $  127  /  hour 

***  According  to  those  two  value,  the  average  value  of  $  96  /  hour  has  been  used  in  this  study  for  Preventive  Maintenance  MMH/FH  cost. 


Table  A. 16: 


DM-1  Alternatives  Scoring  Complete  List 


T-35 


1-  T-35  PILLAN  0,3926 

2-  T-37B  TWEET  0,34553 

3-  T-6A  TEXAN  II  0,34399 

4-  KT-1C  0,33379 

5-  T-34C  0,2804 

6; _ EMB.312 _ 0,28019 


Table  A.  17: 


DM-1  Alternatives  Scoring  Summary  List 


VALUE 

Ergonomics  /  Cockpit 

Systems  Complexity 

Upgradeabllity 

PERFORMANCE 

MAINTAINABILITY 

(UTBU) 

Time  Between  Overhaul  (TBO) 

FLYING  QUALITY 
TECHNOLOGY 

NAVAIDS 

[global  WEIGHT  TOTAL- 


RESULTING  SCORES 


0,2804 1  0,34553 1  0,34399 1  0,28019rq33379^^^1 

T-35 


1-  T-35  PILLAN  0,3926 

2-  T-37B  TWEET  0,34553 

3-  T-6A  TEXAN  II  0,34399 

4-  KT-1C  0,33379 

5-  T-34C  0,2804 

6: _ EMB.312 _ 0,28019 
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Table  A.  18 


DM-2  Alternatives  Scoring  Complete  List 


1-  T-37B  TWEET  0,70748 

2-  T-6A  TEXAN  II  0,581508 

3-  KT-1C  0,57651 

4-  EMB.312  0,51493 

5- 6-  T-35  0,3048 

5-6- _ T-34C _ 0,3048 


Table  A.  19: 


DM-2  Alternatives  Scoring  Summary  List 


i:-  .  IK  A  •••L  ~ 


~ r  ~ n‘j'  ii  ■.  i 


VALUE 


MEASURE 


ALTERNATIVE  SCORES 


WEIGHTED  SCORES 


GLOBAL  WEIGHT  TOTAL- 


RESULTING  SCORES  - 


2|  g(L3Q48^  (L581508  0,51493|  0,57651 1  0,3048| 

T-37B 


T-37B  TWEET 

T-6A  TEXAN  II 

KT-1C 

EMB.312 

T-35 

T-34C 


0,70748 

0,581508 

0,57651 

0,51493 

0,3048 

0.3048 
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Tabic  A. 20 


DM-4  Alternatives  Scoring  Complete  List 


T-6A  TEXAN  II 


1-2-  T-6A  TEXAN  II  0.5296 

1-2-  KT-1C  0,5296 

3-  T-37B  TWEET  0,4938 

4-  EMB.312  0,4608 

5-  T-35  0,3208 

6-  _ T-34C _ 0,3024 


Table  A. 21: 


DM-4  Alternatives  Scoring  Summary  List 


VALUE 

DESIGN  Robustness  /  Landing  Gears 

Ergonomics  l  Visibility 
PERFORMANCE  Power  Loading 

SAFETY  Ejection  Seat 

FLYING  QUALITY  Stability 

TECHNOLOGY_ Consistency  With  Current 


ALTERNATIVE  SCORES 


WEIGHTED  SCORES 


GLOBAL  WEIGHT  TOTAL- 


RESULTING  SCORES 


7\  0,3024  0.4938M  0,5296  0,4608|  0,5208|  0,3208| 


T-6A  TEXAN  II 


1-2-  T-6A  TEXAN  II  0,5296 

1-2-  KT-1C  0,5296 

3-  T-37B  TWEET  0,4938 

4-  EMB.312  0,4608 

5-  T-35  0,3208 

6-  _ T-34C _ 0,3024 
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Table  A. 2 2 


DM-5  Alternatives  Scoring  Complete  List 


Table  A. 23: 


DM-5  Alternatives  Scoring  Summary  List 


VALUE 

DESIGN  Robustness  I  Landing  Gears 

Systems  Complexity 
PERFORMANCE  Thrust 

RELIABILITY  Mean  Time  Between  Failures  (MTBF) 

MAINTAINABILITY  Mean  Time  Between  Maintenance  (MTBM) 

Time  Between  Overhaul  (TBO) 

FLYING  QUALITY  Recoverability 


WEIGHTED  SCORES 


1,000  |  RESULTING  SCORES - =■  |  0,21 37 1  0,3082335^^^^H~lU0982 1  0,3381 627|  0,31S8| 

T-6A  TEXAN  II 


1-  T-6A  TEXAN  II  0,3678194 

2-  KT-1C  0,3381627 

3-  T-35  0,3158 

4-  EMB.312  0,30982 

5-  T-37B  TWEET  0,3082335 

6-  _ T-34C _ 0,2137 
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Table  A. 24: 


DM-6  Alternatives  Scoring  Complete  List 


1-  T-6A  TEXAN  II  0,8151 

2-  KT-1C  0,798015 

3-  T-34C  0,7454 

4-  T-37B  TWEET  0,737731 

5-  EMB.312  0,72579 

6-  _ T-35 _ 0,7007 


Table  A. 25: 


DM-6  Alternatives  Scoring  Summary  List 


PERFORMANCE 
FLYING  QUALITY 

TECHNOLOGY 


Degrees  or  Stability 


ALTERNATIVE  SCORES 


WEIGHTED  SCORES 


GLOBAL  WEIGHT  TOTAL- 


RESULTING  SCORES  - 


~\  0,7454  0,73773T^^^^B~0J2579|  0,798015|  0,7007| 


T-6A  TEXAN  II 


T-6A  TEXAN  II 

KT-1C 

T-34C 

T-37B  TWEET 

EMB.312 

T-35 


0,8151 

0,798015 

0,7454 

0,737731 

0,72579 

0,7007 
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Tabic  A. 26: 


DM-7  Alternatives  Scoring  Complete  List 


T-6A  TEXAN  II 


1-  T-6A  TEXAN  II  0,8482 

2-  KT-1C  0,8234 

3-  T-37B  TWEET  0,7377 

4-  EMB.312  0,69888 

5-  T-34C  0,5662 

6-  _ T-35 _ 0,4754 


Table  A. 27: 


DM-7  Alternatives  Scoring  Summary  List 


T-6A  TEXAN  II 
KT-1C 

T-37B  TWEET 
EMB.312 
T-34C 
T-35 


0,8482 

0,8234 

0,7377 

0,69888 

0,5662 

0,4754 
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Table  A. 28:  DM-8  Alternatives  Scoring  Complete  List 


1-  T-6A  TEXAN  II  0,7526225 

2-  T-37B  TWEET  0,718872 

3-  KT-1C  0,6274225 

4-  EMB.312  0,5638 

5-  T-34C  0,4311 

6-  _ T-35 _ 0,3497 


Table  A. 29: 


DM-8  Alternatives  Scoring  Summary  List 


PERFORMANCE 

MAINTAINABILITY 

FLYING  QUALITY 

TRAINING  QUALITY 


VALUE 

rgonomics  /  Cockpit 


Flight  Path  Stability 


MEASURE 


ALTERNATIVE  SCORES 


GLOBAL  WEIGHT  TC 


RESULTING  SCORES  - 


J  0,431 1  0,71 8872|0, 7526225|  0,5638|  0,6274225|  0,3497| 


T-6A  TEXAN  II 


2 

3 


3 

6 


T-6A  TEXAN  II  0,7526225 

T-37B  TWEET  0,718872 

KT-1C  0,6274225 

EMB.312  0,5638 

T-34C  0,4311 

T-35 _ 0,3497 
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Tabic  A. 30:  DM-9  Alternatives  Scoring  Complete  List 


Table  A. 31: 


DM-9  Alternatives  Scoring  Summary  List 


ALTERNATIVE  SCORES 

WEIGHTED  SCORES 

Global  Weight 

T-34C  IT-37B  IT-6A  IEMB312  IKT-1C  IT-35 

Ergonomics  /  Cochpii 

PERFORMANCE  Endurance 

FLYING  QUALITY  Maneuverability  /  Roll  Rale 

TRAINING  QUALITY  Instrument  Flight 

Low  Level  Flight 

SUPPORTING  SYSTEMS  Synthetic  Training  System 

Debriefing  System 

u 

u 

LI 

1,000 

GLOBAL  WEIGHT  TOTAL - ► 

|  RESULTING  SCORES - ► 

0,5429|  0, 54957|  0,65662|  0,5173T^^^^H)Sl58 

KT-1C 


1-  KT-1C  0,70843 

2-  T-6A  TEXAN  II  0,65662 

3-  T-37B  TWEET  0,54957 

4-  T-34C  0,5429 

5-  EMB.312  0,51731 

6-  _ T-35 _ 0,4158 
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Table  A. 32 


DM-10  Alternatives  Scoring  Complete  List 


1-  T-37B  TWEET  0,6164 

2-  T-6A  TEXAN  II  0,6129463 

3-  KT-1C  0,57145 

4-  EMB.312  0,56499 

5-  T-35  0,5371 

6-  T-34C  0,4993 


Table  A. 33: 


DM-10  Alternatives  Scoring  Summary  List 


[global  WEIGHT  TOTAL - ►  | 


1,000  |  RESULTING  SCORES - ►  1  0,4993H0,6164|  0,6129463|  0,56499|  0,57145|  0,5371 

T-37B 


T-37B  TWEET  0.6164 

T-6A  TEXAN  II  0,6129463 

KT-1C  0,57145 

EMB.312  0,56499 

T-35  0,5371 

T-34C _ 0,4993 
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Table  A. 34:  DM-11  Alternatives  Scoring  Complete  List 


1-  KT-1C  0,70843 

2-  T-6A  TEXAN  II  0,65662 

3-  T-37B  TWEET  0,54957 

4-  T-34C  0,5429 

5-  EMB.312  0,51731 

6-  _ T-35 _ 0,4158 


Table  A. 35: 


DM-11  Alternatives  Scoring 


Summary  List 


ALTERNATIVE  SCORES 

WEIGHTED  SCORES 

VALUE  MEASURE 

Global  Weight 

T-34C  IT-37B  IT-6A  IEMB312  IKT-1C  IT-35 

T-34C  IT-37B  |T-6A  IEMB312  |KT-1C  IT-35 

DESIGN  Ergonomics  i  Cockpit 

FLYING  QUALITY  Stability 

Handling  Quality  Rating 

TECHNOLOGY  Consistency  With  Current 

Degrees  of  Stability 

El 

J  0.33  0,5  0.33  0,5  0.33 

l§  SS 

||  || 

HI! 

1111 

I  ill 

1,000 

GLOBAL  WEIGHT  TOTAL - ► 

|  RESULTING  SCORES - ► 

0,748|  0, 67965|  1  II  |  .  1 

KT-1C 


1-  KT-1C  0,9475 

2-  EMB.312  0,92965 

3-  T-6A  TEXAN  II  0,9125 

4-  T-35  0,7827 

5-  T-34C  0,748 

6-  _ T-37B  TWEET _ 0,67965 
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Table  A. 36:  DM-12  Alternatives  Scoring  Complete  List 


KT-1C 


1-  KT-1C  0.577477 

2-  T-34C  0,5557 

3-  T-37B  TWEET  0,49343 

4-  T-35  0,4838 

5-  T-6A  TEXAN  II  0,457161 

6-  _ EMB.312 _ 0,22912 


Table  A. 3 7: 


DM-12  Alternatives  Scoring  Summary  List 


ALTERNATIVE  SCORES 

WEIGHTED  SCORES 

VALUE  MEASURE 

Glnhal  Weight 

T-34C  IT-37B  IT-6A  IEMB.312  IKT-1C  IT-35 

T.nar  IT.47R  it.ka  IFMR717  irT.ir  it.-w 

DESIGN  Robustness  /  Landing  Gears 

PERFORMANCE  Endurance 

RELIABILITY  Mean  Time  Between  Failures  (MTBF) 

SAFETY  Total  Safety  Incidents  (Class  AB.C) 

FLYING  QUALITY  Maneuverability  f  Roll  Rate 

Handling  Quality  Rating 

SUPPORTING  SYSTEMS  Synthetic  Training  System 

Styling  Star  Value 

Cooper  Harper  Score 

| 

1 

0  0  0.86M  0.5  0,7377  0 

| 

E  \ 

iooo  ! 

)260  1 

1  I 

i  i 

s 

1,000 

GLOBAL  WEIGHT  TOTAL - ► 

|  RESULTING  SCORES - ► 

0,5557  0, 49343|  0,457161  0, 2291210, 5774771  0,4838 

KT-1C 


1-  KT-1C  0,577477 

2-  T-34C  0,5557 

3-  T-37B  TWEET  0,49343 

4-  T-35  0,4838 

5-  T-6A  TEXAN  II  0,457161 

6-  _ EMB.312 _ 0,22912 
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Table  A. 38:  DM-13  Alternatives  Scoring  Complete  List 


KT-1C 


1-  KT-1C  0,72447 

2-  T-6A  TEXAN  II  0,7081041 

3-  T-34C  0,5674 

4-  EMB.312  0,53588 

5-  T-35  0,5298 

6-  _ T-37B  TWEET _ 0,50606 


Table  A. 39: 


DM-13  Alternatives  Scoring  Summary  List 
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Table  A. 40:  DM-14  Alternatives  Scoring  Complete  List 


Table  A. 41: 


DM-14  Alternatives  Scoring  Summary  List 


1-  T-6A  TEXAN  II  0,8012 

2-  KT-1C  0,7985 

3-  T-37B  TWEET  0,6203 

4-  T-34C  0,5642 

5-  EMB.312  0,52435 

6-  _ T-35 _ 0,4507 
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Table  A. 42:  DM-15  Alternatives  Scoring  Complete  List 


[global  WEIGHT  TOTAL - >  | 


1,000  I  resulting  scores - ►  |  |  0,35251  0,42253|  0,376361 1  0,33535|  0,3790748|  0,2877| 

T-37B  TWEET 


1-  T-37B  TWEET  0,42253 

2-  KT-1C  0,3790748 

3-  T-6A  TEXAN  II  0,376361 

4-  T-34C  0,3525 

5-  EMB.312  0,33535 

6-  _ T-35 _ 0,2877 


Table  A. 43: 


DM-15  Alternatives  Scoring  Summary  List 
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Table  A. 44:  Table  of  Critical  Values  of  S  in  the  Kendall  Co¬ 
efficient  of  Concordance 


k 

N 

Addition 

for 

al  values 
N=  3 

3 

5 

6 

7 

s 

Values  at  the  .05  level  of  significance 

3 

64.4 

103.9 

157.3 

9 

54.0 

4 

49.5 

88.4 

143.3 

217.0 

12 

71.9 

5 

62.6 

112.3 

182.4 

276.2 

14 

83.8 

6 

75.7 

136.1 

221.4 

335.2 

16 

95.8 

8 

48.1 

101.7 

183.7 

299.0 

453.1 

18 

107.7 

10 

60.0 

127.8 

231.2 

15 

89.8 

192.9 

349.8 

20 

119.7 

258.0 

468.5 

764.4 

1,158.7 

Values  at  the  .01  level  of  significance 

3 

75.6 

122.8 

185.6 

9 

75.9 

4 

61.4 

109.3 

176.2 

265.0 

12 

103.5 

5 

80.5 

1428 

229.4 

343.8 

14 

121.9 

6 

99.5 

176.1 

282.4 

422.6 

16 

140.2 

8 

66.8 

137.4 

242.7 

388.3 

579.9 

18 

158.6 

10 

85.1 

175.3 

309.1 

494.0 

737.0 

15 

269.8 

475.2 

20 

364.2 

641.2 

mmvmm 
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Table  A. 45:  Table  of  Critical  Values  of  Chi  Square 


Level  of  Significance 


dof 

0.05 

0.025 

0.01 

0.005 

0.001 

1 

3.84 

5.02 

6.63 

7.88 

10.83 

2 

5.99 

7.38 

9.21 

10.60 

13.82 

3 

7.81 

9.35 

11.34 

12.84 

16.27 

4 

9.49 

11.14 

13.28 

14.86 

18.47 

5 

11.07 

12.83 

15.09 

16.75 

20.51 

6 

12.59 

14.45 

16.81 

18.55 

22.46 

7 

14.07 

16.01 

18.48 

20.28 

24.32 

8 

15.51 

17.53 

20.09 

21.95 

26.12 

9 

16.92 

19.02 

21.67 

23.59 

27.88 

10 

18.31 

20.48 

23.21 

25.19 

29.59 

11 

19.68 

21.92 

24.73 

26.76 

31.26 

12 

21.03 

23.34 

26.22 

28.30 

32.91 

13 

22.36 

24.74 

27.69 

29.82 

34.53 

14 

23.68 

26.12 

29.14 

31.32 

36.12 

15 

25.00 

27.49 

30.58 

32.80 

37.70 

16 

26.30 

28.85 

32.00 

34.27 

39.25 

17 

27.59 

30.19 

33.41 

35.72 

40.79 

18 

28.87 

31.53 

34.81 

37.16 

42.31 

19 

30.14 

32.85 

36.19 

38.58 

43.82 

20 

31.41 

34.17 

37.57 

40.00 

45.31 

21 

32.67 

35.48 

38.93 

41.40 

46.80 

22 

33.92 

36.78 

40.29 

42.80 

48.27 

23 

35.17 

38.08 

41.64 

44.18 

49.73 

24 

36.42 

39.36 

42.98 

45.56 

51.18 

25 

37.65 

40.65 

44.31 

46.93 

52.62 
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Figure  A. 2:  DM-1  Value  Hicrarcy 
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Figure  A. 3:  DM- 2  Value  Hierarcy 


Figure  A. 4:  DM-4  Value  Hierarcy 
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Figure  A. 5: 


DM-5  Value  Hierarcy 


Figure  A. 6:  DM- 6  Value  Hierarcy 
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Figure  A.  7:  DM- 7  Value  Hierarcy 


Figure  A. 9: 


DM- 9  Value  Hierarcy 


Figure  A. 10:  DM-10  Value  Hierarcy 


Figure  A. 11:  DM-11  Value  Hierarcy 
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Figure  A. 12: 


DM- 12  Value  Hierarcy 
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CO 


Figure  A. 14:  DM-14  Value  Hierarcy 


Figure  A. 15: 


DM- 15  Value  Hierarcy 


Figure  A. 16:  DM-3  SDVF  for  Avionics  Scale 


Figure  A. 17:  DM-3  SDVF  for  Avionics  Scale 
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V 


Figure  A. 18:  DM-3  SDVF  for  Endurance  in  Minutes 


Figure  A. 19:  DM-3  SDVF  for  MTBF  in  Hours 
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Figure  A. 20:  DM-3  SDVF  for  MTBM  in  Hours 


Figure  A. 21:  DM-3  SDVF  for  TBO  in  Hours 
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Figure  A. 22:  DM-3  SDVF  for  Total  Number  of  Safety  Inci¬ 

dents  (Class  A  &  B  &  C) 


Figure  A. 23:  DM-3  SDVF  for  Degree  of  Stability  Incidents 

(Class  A  &  B  &  C) 
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Appendix  B.  Aircraft  Systems  Terminology 


Airframe  is  any  component  or  structure  that  is  essential  to  the  structural  integrity 
of  the  aircraft.  Even  though  they  aren’t  considered  part  of  the  structural  integrity 
of  the  aircraft,  the  interior  upholstery,  the  aircraft  paint  and  the  static  electricity 
dischargers  are  also  part  of  the  airframe  system. 

Cockpit  instrumentation  is  the  minimum  instrumentation  required  for  general 
aviation  aircraft  flying  under  IFR  conditions  as  defined  in  Federal  Aviation  Reg¬ 
ulations  (FAR)  Part-9:  airspeed  indicator,  altimeter,  magnetic  direction  indicator, 
tachometer  for  each  engine,  oil  pressure  gauge  for  each  engine,  temperature  gauge 
for  each  air-cooled  engine,  oil  temperature  gauge  for  each  air-cooled  engine,  mani¬ 
fold  pressure  gauge  for  each  engine  if  a  variable  pitch  propeller  is  used,  fuel  gauge 
indicating  the  quantity  of  fuel  in  each  tank,  two-way  radio  communications  system 
and  navigational  equipment  appropriate  to  the  ground  facilities  to  be  used,  gyro¬ 
scopic  rate-of-turn  indicator,  slip-skid  indicator,  altimeter  adjustable  for  barometric 
pressure,  clock  displaying  hours,  minutes,  and  seconds,  generator  or  alternator,  gy¬ 
roscopic  pitch  and  bank  indicator  (artificial  horizon),  gyroscopic  direction  indicator, 
and  directional  gyro. 

Aircraft  controls  are  any  component  that  controls  the  aircraft’s  attitude,  head¬ 
ing,  and  altitude  or  changes  the  aerodynamic  characteristics  of  the  aircraft  in  the  air 
or  on  the  ground  (excluding  powerplant).  This  system  is  composed  of  two  primary 
systems:  flight  control  and  ground  control. 
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Powerplant  is  any  component  or  system  that  is  essential  to  developing  thrust 
for  the  aircraft. 

Landing  gear  is  a  subsystem  which  includes  the  wheels,  the  tires,  and  all  asso¬ 
ciated  switches,  controls,  or  systems  for  extending  and  retracting  the  gear.  On  some 
aircraft,  the  extension  and  retraction  of  the  gear  also  requires  a  hydraulic  system. 
However,  this  is  usually  an  independent  system. 

Ground  steering  system  includes  the  rudder  pedals,  any  associated  rods  that 
connect  the  rudder  pedals  to  the  nose  gear,  and  the  steering  collar  on  the  nose  gear 
itself. 

Fuel  system  includes  any  component  that  contributes  to  providing  fuel  through 
the  engine-driven  fuel  pump.  This  includes  any  fuel  lines,  fuel  cutoff  switches,  fuel 
Liters,  tank  switches,  fuel  boost  pumps  (including  the  on/off  switch),  and  fuel  tanks 
(if  integral,  they  are  included  in  both  airframe  and  fuel  system),  and  fuel  tank  related 
equipment  in  the  tanks  except  for  any  fuel  quantity  transmitting  equipment. 

Heating/ Cooling/Ventilation/Pressurization/Environmental  Control  System  (ECS) 
incorporates  all  elements  that  control  the  temperature  or  the  flow  of  air  into  the  cabin. 

This  subsystem  includes  all  hoses  leading  from  the  engine  or  exhaust  systems,  out¬ 
side  air  vent  and  their  respective  plumbing,  and  the  cockpit  controls  to  regulate  the 
temperature.  However,  some  aircraft  are  equipped  with  air-conditioning  systems. 
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Appendix  C.  The  Decision  Makers’  Criteria 


Reliability  is  ’’the  ability  of  an  item  to  perform  a  stated  function  under  stated  con¬ 
ditions,  for  a  stated  period  of  time”  as  defined  in  ISO  8042:  Quality  Vocabulary. 
Reliability,  in  general,  is  the  ability  of  a  person  or  system  to  perform  and  maintain  its 
functions  in  routine  circumstances.  In  statistics,  reliability  is  the  consistency  of  a  set 
of  measurements  or  measuring  instruments.  This  can  either  be  accomplished  whether 
the  measurements  of  the  same  instrument  give  the  same  measurement,  or  two  inde¬ 
pendent  instruments  give  similar  scores.  In  engineering,  reliability  is  the  ability  of  a 
system  or  component  to  perform  its  required  functions  under  stated  conditions  for  a 
specified  period  of  time.  Evaluations  of  reliability  involve  the  use  of  many  statistical 
tools.  Reliability  provides  information  about  the  failure-free  interval.  In  aviation,  the 
reliability  is  quantified  as  mean  time  between  failures  (MTBF)  for  repairable  products 
of  a  system.  It  is  the  operative  average  time  of  a  system  divided  by  the  total  number 
of  errors  which  have  resulted  in  discontinuance  in  the  usage  of  the  system  during  the 
operative  time  and  is  often  attributed  to  the  useful  life  of  the  system.  Calculations  of 
MTBF  assume  that  a  system  is  renewed  or  fixed  after  each  failure  and  then  returned 
to  service  immediately  after  failure.  When  the  MTBF  is  long  compared  to  the  mission 
time,  it  has  perceived  reliability  (i.e.,  few  chances  for  failure).  When  the  MTBF  is 
short  compared  to  the  mission  time,  it  has  perceived  unreliability  (i.e.,  many  chances 
for  failure). 

The  second  criteria,  performance,  is  defined  as  the  examination  of  how  well 
a  plane  meets  its  design  requirements  and  includes  many  aspects  of  the  airplane 
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operation,  such  as  endurance,  thrust,  fuel  efficiency,  speed,  range,  ceiling,  max  take¬ 
off  runway  length,  power  loading,  and  other  elements  of  desirable  performance. 

Endurance  is  a  measure  of  the  time  spent  in  the  air.  Endurance  refers  to  how 
long  (in  time)  the  aircraft  can  fly  for  a  given  amount  of  fuel.  Maximum  endurance 
means  to  fly  for  the  greatest  amount  of  time  for  the  fuel  onboard.  Endurance  is 
sometimes  erroneously  equated  with  range.  The  two  concepts  are  clearly  different. 
Range  is  a  measure  of  distance  flown  while  endurance  is  a  measure  of  time  spent  in 
the  air.  For  example,  a  typical  airplane  may  exhibit  high  endurance  characteristics 
but  poor  range  characteristics. 

Thrust  is  one  of  the  four  aerodynamic  forces  acting  on  an  aircraft  in  flight.  It  is 
the  force  that  pushes  an  aircraft  through  the  air.  Thrust  must  be  greater  than  drag 
to  achieve  the  forward  acceleration  needed  for  takeoff  and  to  increase  an  aircraft’s 
speed  in  level  flight.  Thrust,  like  any  other  force,  is  measured  in  either  newtons  or 
pounds  as  the  unit.  Jet  engines  are  usually  rated  according  to  the  amount  of  thrust 
they  can  produce. 

Fuel  efficiency  is  the  efficiency  of  a  process  that  converts  energy  contained  in  a 
carrier  into  energy  or  work.  In  the  context  of  aviation,  fuel  efficiency  refers  to  the 
energy  efficiency  of  a  particular  aircraft,  where  its  total  output  (range,  mileage  or  km) 
is  given  as  a  ratio  of  range  units  per  a  unit  amount  of  input  fuel.  This  ratio  is  used 
here  in  common  measure  as  gallons  per  flight  hour. 
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Speed  makes  common  sense  to  everybody.  It’s  used  in  knots  as  specified  by 
airworthiness  standards  for  civil  and  military  aircraft  in  the  USA  Federal  Aviation 
Regulations,  which  are  equal  to  one  nautical  mile  per  hour  (1.852  kilometers  per  hour, 
1.1507794  miles  per  hour). 

Range  is  the  distance  an  aircraft  can  fly  between  takeoff  and  landing,  as  lim¬ 
ited  by  fuel  capacity  in  powered  aircraft,  or  cross-country  speed  and  environmental 
conditions  in  unpowered  aircraft. 

Ceiling  is  the  maximum  usable  altitude  of  an  aircraft.  Specifically,  it  is  the 
density  altitude  at  which  flying  in  a  clean  configuration  (the  flight  configuration  of  an 
airplane  when  its  external  equipment,  such  as  wing  flaps,  landing  gear,  slats,  spoilers 
etc.  are  retracted),  at  the  best  rate  of  climb  airspeed  for  that  altitude  and  with  all 
engines  operating  and  producing  maximum  continuous  power  will  produce  a  100  feet 
per  minute  climb  rate. 

Take-off  runway  length  is  the  horizontal  distance  along  the  takeoff  path  from 
the  start  of  the  takeoff  to  the  point  at  which  the  aircraft  attains  and  remains  at  least 
35  feet  above  the  takeoff  surface,  attains  and  maintains  a  speed  of  at  least  take-off 
safety  speed  and  establishes  a  positive  rate  of  climb. 

Power  loading  (power-to-weight  ratio)  is  a  measure  commonly  used  when  com¬ 
paring  various  vehicles,  including  automobiles,  motorcycles  and  aircraft  (usually  air¬ 
craft  engines).  It  is,  simply,  the  power  the  engine  develops,  divided  by  the  vehicle’s 
weight.  In  an  aircraft,  any  additional  weight  requires  more  lift  to  be  generated  by  the 
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wings  in  order  to  lift  it.  More  lift  from  the  wings  automatically  means  more  drag, 
through  a  process  known  as  induced  drag,  slowing  the  plane  down.  Thus  if  any  two 
engines  deliver  the  same  power,  the  lighter  one  will  result  in  a  faster  plane.  Power-to- 
weight  ratio  therefore  has  a  major  impact  on  overall  performance  in  aircraft,  including 
maximum  speed.  In  this  usage,  the  power-to- weight  ratio  is  typically  used  to  refer  to 
the  weight  of  the  engine  alone  as  a  useful  way  of  comparing  various  aircraft  engines. 
The  term  applied  to  the  aircraft  as  a  whole  is  power  loading. 

Third  criteria,  maintainability,  as  defined  in  Wikipedia,  is  a  characteristic  of 
design  and  installation  expressed  as  the  probability  that  an  item  will  be  retained  in  or 
restored  to  a  specified  condition  within  a  given  period  of  time,  when  the  maintenance 
is  performed  in  accordance  with  prescribed  procedures  and  resources.  Maintainability 
measures  the  ease  and  speed  with  which  a  system  can  be  restored  to  operational  status 
after  a  failure  occurs.  Maintainability  predictions  enable  the  repair  metrics  (MTBM 
and  TBO)  of  the  systems  be  analyzed. 

As  explained  in  NATO  Reliability,  Maintainability,  and  Availability  (RM  and 
A)  Parameters  and  Methods  of  Calculation  document  (Appendix  C),  MTBM  mea¬ 
sures  the  mean  time  between  unscheduled,  line  or  organizational  maintenance  actions 
caused  by  a  design  or  manufacturing  defect,  with  time  expressed  as  total  production 
aircraft  flight  hours.  This  measure  includes  inherent  maintenance  actions  (failures 
which  result  from  an  internal  cause),  unscheduled  maintenance,  and  time  in  flying 
hours  (production  aircraft). 


140 


Time  between  overhaul  (TBO),  a  time  recommended  by  the  manufacturer,  is 
one  important  measure  of  an  aircraft  engine’s  overall  economics  and  is  how  often 
it  has  to  be  overhauled.  The  time  between  overhauls  is  generally  a  function  of  the 
complexity  of  the  engine.  Piston-based  engines  are  much  more  complex  than  turbine- 
powered  engines,  and  generally  have  TBO’s  of  1600  to  2000  hours  of  running  time. 
Since  the  overhaul  process  requires  the  engine  to  be  taken  apart,  it  is  typically  an 
expensive  process. 

Increased  aircraft  maintainability  results  in  shorter  downtime,  reduced  removal 
and  replacement  time,  increased  troubleshooting  capability,  and  less  scheduled  main¬ 
tenance.  Several  decision  makers  also  considered  the  maintenance  specialty  require¬ 
ment,  civilian  airports  cross  service  capability,  and  recording  capability  of  the  system 
as  some  important  inputs  to  the  maintainability  of  the  system. 

Maintenance  specialty  requirement  is  explained  as  when  the  aircraft  requires 
any  specialty  maintenance  in  case  of  landing  at  another  airfield  which  is  either  a 
civilian  or  another  kind  of  aircraft  base.  The  levels  of  that  maintenance  requirement 
are  all  specialty-certificate  maintenance  requirement,  no  safety- flight  issues  mainte¬ 
nance  requirement,  and  no  specialty-certificate  maintenance  requirement.  Those  spe¬ 
cialty  categories  include  aircraft  electronics  (avionics),  composite  structural  repair, 
nondestructive  inspection  and  metal  structures  repair.  If  the  aircraft  is  all  specialty- 
certificate  maintenance  required,  in  case  of  a  breakdown  in  a  cross  country  mission, 
the  home  base  maintenance  team  arrives  at  that  airfield  in  order  to  supply  the  air¬ 
craft  with  the  special  procedures  required.  If  the  aircraft  has  no  safety-flight  issues 
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maintenance  required,  the  general  inspection  and  the  first  overhaul  may  be  done  by 
the  maintenance  team  of  that  airfield,  and  the  aircraft  can  fly  back  to  its  main  base 
with  the  discrepancy  causing  no  problem  for  the  return  sortie.  If  the  aircraft  has  no 
specialty-certificate  maintenance  requirement,  the  entire  repair  can  be  done  by  the 
maintenance  team  of  that  airfield  where  the  aircraft  broke  down. 

Cross-servicing  capability  is  defined  by  NATO  Logistics  Handbook  (October, 1997) 
as  services  performed  on  an  aircraft  by  an  organization  other  than  that  to  which  the 
aircraft  is  assigned,  according  to  an  established  operational  aircraft  cross-servicing 
requirement,  and  for  which  there  may  be  a  charge.  Aircraft  cross-servicing  is  divided 
into  two  categories: 

Stage  A  Cross-Servicing:  The  servicing  of  an  aircraft  on  an  aerodrome/ship 
which  enables  the  aircraft  to  be  flown  on  another  mission,  without  change  to  the 
weapon  configuration.  The  servicing  includes  the  installation  and  removal  of  weapon 
system  safety  devices,  refueling,  replenishment  of  fluids  and  gases,  drag  chutes  packing 
facilities  and  ground  handling. 

Stage  B  Cross- Servicing:  The  servicing  of  aircraft  on  aerodromes/ships  which 
enables  the  aircraft  to  be  flown  on  an  operational  mission.  The  servicing  includes  all 
Stage  A  services  plus  the  loading  of  weapons  and/or  film/ videotape  and  the  replen¬ 
ishment  of  chaff  and  flares.  This  includes  the  processing  and  interpretation  of  any 
exposed  film/videotape  from  the  previous  mission. 
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Data  recording  capacity  aids  both  to  maintenance  team  for  diagnosing  and  de¬ 
termining  any  kind  of  discrepancy  happening  in  flight  and  flight  safety  inspectors  in 
order  to  determine  the  cause  of  the  accident  in  case  of  a  mishap.  The  flight  data 
recorder  (FDR)  records  various  flight  parameters  such  as  engine  status,  fuel  status, 
airspeed,  altitude,  attitude  and  control  settings.  The  cockpit  voice  recorder  (CVR) 
typically  records  the  voice  communications  of  the  crew  members,  although  it  may  also 
pick  up  other  extraneous  noise  in  the  cockpit  such  as  an  explosion.  The  FDR  and 
CVR  have  usually  been  provided  in  the  form  of  on-board  tape  or  disk  recorders.  They 
are  typically  packaged  in  an  armored  casing,  sometimes  called  a  black  box,  which  is 
located  in  the  tail  section  of  the  aircraft.  This  gives  them  the  greatest  chance  of 
surviving  a  crash  and  yielding  their  recorded  data  for  analysis  by  experts. 

The  fourth  criteria,  aircraft  technology,  includes  quite  a  wide  area  of  study. 
Aviation  in  general  and  military  aviation  in  particular  are  driven  by  technology.  Tech¬ 
nologies  are  becoming  outdated  faster  than  the  full  utilization  of  aircraft.  What  is 
meant  here  is  the  technology  level  of  the  electronic  systems,  such  as  navigation  aids, 
radar,  and  communication  system  etc,  installed  on  the  aircraft  and  the  consistency 
and  compatibility  of  those  systems  to  the  state-of-the-art  systems  currently  in  use. 

The  fifth  criteria,  safety,  is  one  of  the  main  considerations  in  design,  acquisition 
and  operational  process  of  an  aircraft.  Safety  includes  all  elements  of  design,  manufac¬ 
turing,  operation,  maintenance,  servicing  and  infrastructure  support  that  affect  safe 
accomplishment  of  the  flight  mission.  However,  this  study  focuses  on  the  emergency 
aspects  of  flight  operation,  aircraft  maintenance,  and  airworthiness  of  the  aircraft. 
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According  to  a  briefing  report  (Military  Aircraft  Safety,  1998)  prepared  by  the  US 
General  Accounting  Office,  safety  incidents  (mishaps)  involve  any  reportable  damage 
to  an  aircraft  that  is  preparing  to  fly,  in  flight,  or  completing  a  landing.  Safety 
incidents  are  classified  by  DoD  according  to  the  severity  of  the  resulting  injury  or 
property  damage.  Class  A  mishaps  involve  damage  of  $1  million  or  more,  a  destroyed 
aircraft,  or  a  fatality  or  permanent  total  disability.  The  remaining  classes  of  mishaps 
are  distinguished  primarily  by  their  loss  of  value  and  severity  of  injury.  Class  B 
accidents  involve  damage  ranging  from  $200,000  to  less  than  $1  million,  permanent 
partial  disability,  or  inpatient  hospitalization  of  five  or  more  people.  Class  C  accidents 
involve  damage  ranging  from  $10,000  to  less  than  $200,000  or  a  lost-time  injury.  And 
Class  D  accidents  involve  damage  of  less  than  $10,000.  Here,  the  3  major  incident 
classes  have  been  taken  into  consideration  as  the  major  safety  indicator. 

Military  aircraft  are  designed  with  some  on-board  emergency  systems  such  as 
ejection  seat,  hook  and  drag  chute.  The  ultimate  technology  in  ejection  seat  use  is 
a  0-Feet/0-Knot  ejection  seat  which  is  designed  to  safely  extract  upward  and  land 
its  pilot  from  a  grounded  stationary  position,  specifically  from  aircraft  cockpits.  The 
zero-zero  capability  was  developed  to  help  aircrew  escape  upward  from  unrecoverable 
emergency  situations  during  low  altitude  and/or  low  speed  flight  as  well  as  ground 
mishaps.  The  other  ejection  seats  have  to  be  used  at  minimum  altitudes  and  airspeeds 
(in  a  flight  envelope).  In  addition,  military  aircraft  also  have  drag  chute  and  hook 
systems  to  be  used  in  case  of  a  brake  failure  or  difficulty  in  stopping  the  aircraft  in  the 
remaining  length  of  the  runway.  Finally,  the  collision  avoidance  systems  contribute 
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to  the  safety  criteria  by  providing  the  pilot  with  situational  awareness  in  any  flight 
condition.  TCAS  (Traffic  Collision  Avoidance  System)  is  a  computerized  avionics 
device  which  is  designed  to  reduce  the  danger  of  mid-air  collisions  between  aircraft 
by  monitoring  the  airspace  around  an  aircraft,  independent  of  air  traffic  control,  and 
warning  pilots  of  the  presence  of  other  aircraft  which  may  present  a  threat  of  mid-air 
collision.  GCAS  (Ground  Collision  Avoidance  System)  is  a  warning  system  alerting 
the  pilot  if  his  aircraft  is  in  immediate  danger  of  flying  into  the  ground,  monitors  the 
aircraft’s  height  above  ground  as  determined  by  a  radar  altimeter,  and  has  a  computer 
that  keeps  track  of  these  readings,  calculates  trends,  and  warns  the  pilot  with  visual 
and  audio  messages  if  the  aircraft  is  in  certain  defined  flying  configurations. 

The  sixth  criteria,  flying  quality,  includes  different  aerodynamic  design  con¬ 
siderations  like  recoverability,  stability,  maneuverability,  handling  quality  rating  and 
flight  path  stability. 

Recoverability  in  this  study  is  the  term  used  for  the  potential  to  keep  the  aircraft 
in  the  controlled  margin  of  the  flight  envelope.  The  stall  warning  before  the  stall 
margin  is  the  measure  of  recoverability  in  Knots.  Flying  qualities  are  reported  using 
qualitative  comments  and  the  Cooper-Harper  Rating  Scale  and/or  the  Pilot  Induced 
Oscillation  Rating  Scale  where  appropriate  tasks  are  defined. 

Stability  is  the  aircraft’s  response  when  disturbed  from  a  given  angle  of  attack, 
slip  or  bank.  There  are  4  different  types  of  stability:  totally,  normally  (positively), 
neutrally  (dynamically),  and  negatively  (fully  unstable). 
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Totally  stable:  Aircraft  will  return,  more  or  less  immediately,  to  its  trimmed 
state  without  pilot  intervention;  however,  such  an  aircraft  is  rare  and  undesirable. 
We  usually  want  an  aircraft  just  to  be  reasonably  stable  so  it  is  easy  to  fly.  If  it  is  too 
stable,  it  tends  to  be  sluggish  in  maneuvering  and  heavy  on  the  controls.  If  it  tends 
toward  instability,  the  pilot  has  to  continually  watch  the  aircraft’s  attitude  and  make 
the  restoring  inputs,  which  becomes  tiring,  particularly  when  flying  by  instruments. 
Some  forms  of  instability  make  an  aircraft  unpleasant  to  fly  in  marginal  weather. 

Normally  (positively)  stable:  Aircraft,  when  disturbed  from  its  trimmed  flight 
state,  will  commence  an  initial  movement  back  towards  the  trimmed  flight  state  (with¬ 
out  pilot  intervention),  but  it  then  starts  a  series  of  damping  oscillations  about  the 
original  flight  state.  This  damping  process  is  usually  referred  to  as  dynamic  stability, 
and  the  initial  movement  back  towards  the  flight  state  is  called  static  stability.  The 
magnitude  of  the  oscillation  and  the  time  taken  for  the  oscillations  to  completely 
damp  out  is  another  aspect  of  stability. 

Neutrally  (dynamically)  stable:  Aircraft  will  continue  oscillating  after  a  distur¬ 
bance,  but  the  magnitude  of  those  oscillations  will  neither  diminish  nor  increase.  If 
these  were  oscillations  in  pitch,  the  aircraft  will  just  continue  porpoising. 

Negatively  stable  (fully  unstable):  Aircraft  may  be  statically  unstable  and  never 
attempt  to  return  towards  the  trimmed  state.  Or  it  can  be  statically  stable  but 
dynamically  unstable,  where  it  will  continue  oscillating  after  disturbance  with  the 
magnitude  of  those  oscillations  getting  larger  and  larger.  Significant  instability  is  an 
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undesirable  characteristic,  except  where  an  extremely  maneuverable  aircraft  is  needed 
and  the  instability  can  be  continually  corrected  by  on-board  fly-by-wire  computers 
rather  than  the  pilot.  For  example,  a  supersonic  air  superiority  fighter.  The  best 
piston-engined  WWII  day  fighters  were  generally  designed  to  be  just  stable  longitu¬ 
dinally,  neutrally  stable  laterally  and  positively  stable  directionally. 

Maneuverability  is  determined  by  the  aircraft’s  ability  to  change  attitude  and 
velocity  around  its  three  axes  (longitudinal,  lateral,  and  vertical).  Changes  in  attitude 
on  each  axis  are  created  by  the  moments  acting  on  the  aircraft  on  that  axis  and  are 
resisted  by  the  mass  moments  of  inertia  of  the  aircraft  on  that  axis.  Changes  in 
velocity  on  each  axis  are  created  by  the  forces  acting  on  that  axis  and  are  resisted  by 
the  mass  inertia  (mass,  or  weight)  of  the  aircraft.  Maneuver  capacity  of  an  aircraft  is 
measured  by  roll  rate  and  g-capacity. 

Handling  qualities  may  be  defined  as  dynamic  and  static  properties  of  an  aircraft 
that  permit  the  pilot  to  fully  use  its  performance  in  a  variety  of  missions  and  roles. 
Handling  quality  is  measured  using  the  Cooper-Harper  rating  and  done  subjectively  by 
the  human  pilot.  The  Cooper-Harper  rating,  a  set  of  criteria  to  evaluate  the  handling 
qualities  of  aircraft,  has  been  taken  as  a  standard  for  measuring  the  performance  of 
aircraft  since  it  was  introduced  in  1966.  Aircraft  performance,  ability  to  control  the 
aircraft,  and  the  degree  of  pilot  compensation  needed  are  three  major  key  factors 
used  in  deciding  the  aircraft  handling  qualities  in  the  Cooper-Harper  rating.  The 
Cooper-Harper  rating  scale  ranges  from  1  to  10,  with  1  indicating  the  best  handling 
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characteristics  and  10  the  worst.  The  criterion  is  evaluative  and  thus  the  scale  is 


considered  subjective. 

Flight  path  stability  is  basically  a  measure  of  an  aircraft’s  ability  to  maintain  the 
glidepath  at  a  defined  approach  speed.  Flight  path  stability  is  the  slope  of  the  flight 
path  angle  versus  the  velocity  curve  at  approach  speed.  Since  flight  path  stability 
affects  an  aircraft’s  handling  qualities  during  final  approach,  the  military  uses  it  as  a 
criterion  for  power  approach  and  landing  handling  qualities.  It’s  measured  in  degrees 
per  knot. 

Seventh  criteria,  training  quality,  is  the  evaluation  of  different  phases  of  the 
flight  training.  This  training  is  composed  of  basic  handling,  aerobatics,  instrument 
flight,  formation  flight,  night  flight,  low-level  flight  and  ground  handling.  All  these 
phases  have  been  mentioned  by  the  decision  makers  except  basic  handling  and  night 
flight. 

Aerobatics  is  the  demonstration  of  flying  maneuvers  for  training.  Many  aero¬ 
batic  maneuvers  involve  rotation  of  the  aircraft  about  its  longitudinal  axis  (rolling) 
or  the  pitch  axis  (looping).  Some  complex  maneuvers,  such  as  a  spin,  also  require 
that  the  aircraft  be  displaced  around  a  vertical  axis,  known  as  yawing.  Maneuvers 
are  often  combined  which  demand  a  higher  level  of  skill  from  the  pilot,  but  greatly 
increase  the  performance  of  an  aerobatic  flight  sequence.  The  aerobatic  maneuvers 
mentioned  below  are  in  the  order  of  their  difficulty  beginning  from  the  easier  to  the 
hardest:  Chandelle,  hammerhead,  loop,  immclman,  cuban-8  and  spin.  Aerobatics  are 
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taught  to  the  military  pilots  as  a  means  of  developing  precise  flying  skills  and  for 
tactical  use  in  combat. 

Chandelle  is  a  maneuver  designed  to  show  the  pilot’s  proficiency  in  controlling 
the  aircraft  while  performing  a  minimum  radius  climbing  turn  at  a  constant  rate  of 
turn  (expressed  usually  in  degrees  per  second)  through  a  180  degree  change  of  heading, 
arriving  at  the  new  reciprocal  heading  at  an  airspeed  in  the  slow-flight  regime,  very 
near  to  the  aerodynamic  stall. 

Hammerhead  is  essentially  an  aeronautical  cartwheel.  The  maneuver  begins 
from  a  horizontal  line  pulling  the  airplane  up  smoothly  but  aggressively  to  establish  a 
vertical  line  and  hold  the  vertical  line  until  the  airplane  almost  runs  out  of  airspeed, 
and  just  at  that  point,  pushing  full  left /right  rudder  to  make  the  airplane  pivot,  or 
cartwheel,  around  its  left/right  wing,  then  establishing  and  holding  a  vertical  dive 
before  pulling  the  nose  back  up  to  a  horizontal  line.  The  hammerhead  ends  with  the 
airplane  flying  180  degrees  from  its  original  heading. 

Loop  is  a  vertical  circle  entered  from  straight  and  level  flight.  A  positive  pitching 
movement  is  used  at  all  points  in  the  loop  to  draw  the  circle,  so  that  the  aeroplane 
canopy  is  pointing  inwards. 

Immclman  is  a  half  loop  followed  by  a  half  roll.  There  should  be  no  pause 
between  the  end  of  the  loop  and  the  start  of  the  roll  to  upright  flight.  Cuban-8  is 
a  combination  of  2  loops  and  2  rolls.  It  starts  with  5/8s  of  a  loop  to  the  45  degree 
nose  down,  1/2  roll  at  that  point,  and  then  another  5/8s  of  a  loop  to  the  45  degree 
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nose  down,  1/2  roll  to  the  upright  position  and  completes  the  maneuver  with  3/8s  of 
a  loop  to  level  flight. 

Spin  is  an  aggravated  stall  resulting  in  rotation  wherein  the  aircraft  follows  a 
downward  and  winding  path.  Spins  can  be  entered  unintentionally  or  intentionally, 
from  any  flight  attitude  and  from  practically  any  airspeed.  The  only  thing  required 
is  sufficient  yaw  at  the  moment  an  aircraft  stalls. 

Instrument  flight  has  two  phases  coming  one  after  another:  Basic  instrument 
flight  training  and  radio  instrument  flight  training.  Basic  instrument  flight  training 
provides  an  introduction  to  basic  instrument  flight  rules  (IFR)  procedures  and  regula¬ 
tions,  an  introduction  to  airplane  instruments  and  instrument  flying  techniques,  IFR 
airspace  and  air  traffic  control  procedures,  IFR  weather  and  weather  services,  decision 
making  in  instrument  conditions,  proper  instrument  crosscheck  and  interpretation, 
and  IFR  flight  planning.  Basic  instrument  flight  maneuvers  are  straight  and  level 
flight  (pitch,  bank,  and  power  relationship,  trim  and  control  usage),  turns  (speed, 
bank  angle,  and  standard  rate  turn  relationship,  constant  rate,  standard  and  half- 
standard  turns,  bank  control  during  roll-in  and  out,  coordination),  airspeed  changes 
(using  trim  and  altimeter),  climbs  and  descents  (constant  airspeed  vs  constant  rate, 
level-off  techniques,  entry  procedures,  power-FPM  relationship),  steep  turns  (cross¬ 
check,  coordination  during  roll-in,  turn,  and  roll-out,  priority  of  the  attitude  indica¬ 
tor)  within  the  limits  of  altitude  (100  feet),  heading  (10),  airspeed  (10  knots),  and 
bank  (5)  as  determined  by  Federal  Aviation  Administration  (FAA)  Instrument  Rat¬ 
ing  Practical  Test  Standards  (2004).  Radio  instrument  flight  training  provides  all  the 
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knowledge  about  basic  enroute  radio  navigation  procedures  for  SIDs  (Standard  Instru¬ 
ment  Departures),  radial/bearing  intercepts,  radial  intercept,  arcing,  point-to-point 
navigation,  correction  for  wind  drifts,  ground  speed  checks,  time/distance  checks, 
voice  procedures,  direct  routing,  indications  of  station  passage  on  the  VOR-TACAN/ 
VOR  /  VOR-DME,  flying  a  TACAN  /  VOR-DME  arc,  instrument  approach  holding 
procedures,  instrument  landing  system  (ILS)  approach  procedures,  circling  approach 
procedures,  missed  approach  procedures,  climb/cruise/descend  profiles,  emergency 
procedures  in  IFR  flight  and  so  on.  If  the  aircraft  is  equipped  with  the  required  train¬ 
ing  materials  (hood  or  similar  systems),  both  the  basic  and  radio  instrument  flight 
training  can  be  given  in  full  IFR  conditions  which  disables  the  student  to  get  any 
kind  of  help  from  the  outside  visual  references  to  align  his  spatial  orientation. 

Formation  flight  is  a  disciplined  flight  of  two  or  more  aircraft,  which  requires 
attitude,  focus,  and  practice,  under  the  command  of  a  flight  leader,  using  a  stan¬ 
dardized  set  of  signals  and  commands  to  direct  the  wingmen.  Formation  flight  is 
derived  from  the  military  need,  mostly  tactical  in  nature,  for  protection  and  strike 
capability.  Since  it  is  a  kind  of  special  flight  requiring  utmost  coordination  of  speed 
and  aircraft  handling,  the  formation  flight  is  evaluated  according  to  the  altitude  and 
speed  regimes,  the  lower  the  altitude,  the  harder  to  coordinate  and  accomplish  the 
other  checks. 

Aircraft’s  ground  handling  is  accomplished  by  the  nosewheel  steering  system. 
To  steer  the  aircraft  on  the  ground,  the  pilot  uses  the  nose-gear  steering  button  in 
the  cockpit  to  activate  the  system  together  with  the  rudders.  During  take-off  and 
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landing,  directional  control  of  the  aircraft  is  maintained  solely  by  the  rudder.  If  the 
pilot  wants  to  turn  right/left,  s/he  simply  turns  the  steering  wheel  in  the  desired 
direction.  When  no  input  is  applied  to  the  steering  wheel,  the  selector  valve  returns 
to  the  null  point  and  allows  the  nose  wheel  to  pivot  freely  as  differential  braking  is 
applied.  Another  steering  system  is  the  castering  nosewheel.  The  castering  nosewheel 
isn’t  steerable  in  the  conventional  sense,  ft  swivels  freely,  90  to  the  left  and  right; 
steering  is  by  differential  braking.  This  arrangement  allows  aircraft  to  turn  tighter 
than  planes  with  conventional  nosewheels.  By  applying  brakes  to  only  one  side,  it’s 
possible  to  get  the  aircraft  to  spin  around  a  wheel  like  a  taildragger. 

The  eighth  criteria,  design,  comprises  a  large  number  of  design  subcriteria  allow¬ 
ing  the  aircraft  to  be  handled  much  more  efficiently  and  comfortably.  These  subcri¬ 
teria  are  robustness,  simplicity,  ergonomics,  systems  complexity,  system  dependency, 
user  friendly,  upgradeability,  and  deactivation  capability. 

Robustness  of  the  landing  gear  and  airframe  is  one  of  the  most  important  re¬ 
quirements  for  a  training  aircraft.  The  landing  gear  is  a  vital  and  robust  component 
of  the  aircraft,  for  which  there  is  no  built-in  redundancy.  This  sophisticated  assem¬ 
bly  must  ensure  that  the  shock  of  a  very  bad  landing,  especially  by  a  student  pilot, 
can  be  reliably  withstood  without  any  problems.  A  good  measure  of  robustness  of 
the  landing  gear  is  the  failure  rate  per  100,000  flight  hours.  Flying  hours  play  the 
dominant  role  in  the  failure  rate  estimates.  The  service  life  of  the  airframe  is  another 
important  factor  taken  into  consideration  during  the  procurement  phase.  According 
to  a  study  report  (Aircraft  Historical  and  Future  Developments)  carried  by  World 
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Meteorological  Organization  (WMO)  and  United  Nations  Environment  Programme 
(UNEP),  average  service  life  of  an  airframe  is  25  to  35  years.  The  airframe  exper¬ 
iments,  which  are  designed  to  demonstrate  airframe  structural  reliability,  airframe 
turn  time,  and  airframe  lifetime,  are  accomplished  by  simulating  the  essential  load¬ 
ing  events  the  airframe  can  expect  to  see  during  an  entire  mission  cycle,  and  then 
repeating  the  cycle.  All  significant  airframe  processing  issues  such  as  fuel  fill  and 
drain  cycles,  repair  and  replacement  of  thermal  protection  system  components,  and 
the  integrity/  repair  /  replacement  of  thermal  and  or  other  are  considered. 

Simplicity  of  cockpit  design  and  the  emergency  procedures  provide  an  evident 
contribution  in  the  reaction  pace  of  the  pilots  in  case  of  accidents  and  incidents. 
Because  of  this,  over  the  years,  some  aircraft  are  considered  safer,  with  lower  risk 
compared  to  others  based  on  accident  statistics.  That  statistic  is  directly  related  to 
design  features  of  the  airplane.  Basic  6-pack  cockpit  configuration  is  an  array  of  6 
essential  gauges  arranged  in  2  rows  of  3,  directly  in  front  of  the  pilot.  In  clockwise 
order  from  the  upper  left  are  found  the  airspeed  indicator,  artificial  horizon,  altimeter, 
vertical  speed  indicator,  heading  display,  and  turn  coordinator.  Uniform  positioning 
of  instrumentation  makes  it  easy  for  the  pilot  to  transition  from  one  type  of  aircraft  to 
another  and  minimizes  the  amount  of  time  required  for  airborne  flight  crews  to  obtain 
situation  awareness  in  dynamically  changing  environments.  Even  in  modern  commer¬ 
cial  aircraft  equipped  with  glass  cockpits  (computer  displays  instead  of  gauges),  a 
computer-generated  representation  of  these  6  gauges  is  often  displayed  on  one  of  the 
screens.  Standardized  cockpits  allow  the  aircrew  to  develop  a  spatial  orientation  to 
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information.  Aircrew  can  develop  a  scan  pattern  habit  that  is  well  defined  and  prac¬ 
ticed  so  that  data  are  transferred  consistently  and  efficiently  from  the  controls  and 
displays  for  the  aircrew  to  accomplish  their  mission  safely  and  effectively.  This  inher¬ 
ent  standardization  also  aids  the  crew  in  the  coordination  of  tasks  and  responsibilities 
to  manage  mission  workload.  Another  significant  benefit  to  standardization  between 
cockpit  designs  is  that  it  should  improve  pilot  performance,  reduce  pilot  errors,  and 
reduce  task  performance  times  when  transitioning  between  different  aircraft  systems. 

Ergonomics  of  the  cockpit  is  more  than  just  a  convenience,  ft  also  helps  counter 
tiredness  and  fatigue  during  long  flights.  Comfort  is  thereby  an  important  contribu¬ 
tion  to  both  performance  and  active  safety.  The  cockpit  should  be  able  to  be  adjusted 
to  perfectly  fit  almost  any  pilot.  The  seat  bottom  should  move  up  and  forward  for 
smaller  pilots,  thus  always  keeping  the  pilot  on  the  same  horizontal  plane,  and  making 
sure  that  the  stick  is  always  in  the  optimal  position  relative  to  her  or  his  hands.  The 
rudder  pedals  should  move  away  from  each  other  when  adjusted  for  shorter  pilots, 
this  avoids  the  well  known  problem  of  shorter  pilots  only  reaching  the  outer  parts  of 
the  pedals  when  flying  an  aircraft  with  a  centrally  mounted  instrument  console. 

Visibility  is  one  of  the  main  topics  of  the  aviation  studies  since  accidents  in 
degraded  visibility  continue  to  account  for  a  disproportionately  large  number  of  fatal 
crashes  in  general  aviation.  Most  aircraft  cockpits  severely  limit  the  held  of  view  avail¬ 
able  to  the  pilot.  Visibility  is  most  restricted  on  the  side  of  the  aircraft.  Any  aircraft 
characteristics  relating  to  cockpit  visibility,  proper  eye  height,  seat  position  or  in¬ 
strument  lighting  intensities  related  to  transition  through  areas  of  varying  brightness, 
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visual  conditions  can  also  affect  a  pilot’s  spatial  orientation.  The  different  cockpit 
types  for  the  visibility  mentioned  in  this  study  are  binocular  cockpit  visibility  (B737), 
clam-shell  type  canopy  (T-37),  shoulder  level  canopy  (F-4),  bubble  canopy  (P-47)  and 
bubble  canopy  with  the  transparency  (F-16)  which  allows  the  same  clarity  in  vision 
over  the  entire  canopy  because  of  the  components  within  the  canopy  material. 

Noise  level  in  the  cockpit  is  another  consideration  since  the  pilot’s  hearing  may 
be  injured  is,  a  common  problem.  The  noise  level  was  measured  in  the  cockpit  in  a 
study  carried  out  by  Wu  and  Ding  [39].  The  temporary  threshold  shift  (TTS)  was 
studied  in  20  healthy  young  men,  and  permanent  threshold  shift  was  examined  in 
166  fighter  pilots.  The  results  showed  that  noise  level  in  the  cockpit  was  110  dBA 
and  TTS  after  2  min  noise  exposure  decreased  significantly  and  reached  13  dB  at  a 
particular  frequency.  It  was  also  found  that  56%  of  the  166  pilots  suffered  from  high 
frequency  hearing  loss,  and  the  percentage  increased  with  flight  time.  This  indicates 
that  cockpit  noise  may  cause  a  permanent  threshold  shift  of  hearing. 

Environmental  control  systems  (ECS)  provide  both  heating  and  air  condition¬ 
ing  to  the  cockpit  and  nose  bay  environments,  cooling  down  or  heating  a  given  amount 
of  air  for  the  comfort  of  the  aircrew  and  the  safety  of  electrical  equipment.  Changes 
in  atmospheric  conditions  (temperature,  pressure,  altitude,  relative  humidity)  neces¬ 
sitate  the  careful  design  of  environmental  control  systems.  Monitoring  air  flow  in 
the  aircraft,  ECS  is  critical  to  ensure  the  adequate  cooling  of  critical  areas,  such  as, 
the  avionics.  As  aircraft  became  capable  of  obtaining  altitudes  above  that  at  which 
flight  crews  could  operate  efficiently,  a  need  developed  for  complete  environmental 
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systems.  Air  conditioning  provides  the  proper  temperature  and  supplemental  oxy¬ 
gen  which  could  provide  sufficient  breathable  air.  One  problem  was  that  not  enough 
atmospheric  pressure  exists  at  high  altitude  to  aid  in  breathing,  and  even  at  lower 
altitudes  the  body  must  work  harder  to  absorb  sufficient  oxygen  through  the  lungs 
to  operate  at  the  same  level  of  efficiency  as  at  sea  level.  This  problem  was  solved  by 
pressuring  the  cockpit/cabin  area. 

Increasing  system  complexity  of  aircraft  technology  coupled  with  requirements 
to  operate  at  very  low  level  and  in  all  weather  conditions  creates  a  heavy  workload  in 
military  cockpits,  especially  in  single  seat  aircraft  (NATO  Case  Studies  and  Future 
Applications,  RTO-TR-IST-037).  A  pilot’s  top  priority  should  be  to  fly  the  aircraft, 
which  requires  the  use  of  his  hands  and  eyes.  The  operation  of  other  equipment, 
although  necessary  for  the  mission,  may  be  a  distraction  from  the  primary  task.  A 
study  carried  out  by  Essential  Skills  Research  Unit  Skills  and  Labor  Market  Informa¬ 
tion  Division  Skills  and  Employment  Branch,  Canada,  defines  the  complexity  levels 
as  follows:  Many  essential  skills  have  been  given  two  types  of  ratings  for  complexity 
-  a)  the  range  of  complexity  of  typical  tasks  for  the  occupation  (uncomplicated-  a 
task  which  is  typical  and  occurs  frequently  in  the  job  or  occurs  less  frequently,  but 
nevertheless  is  required  by  virtually  all  incumbents),  and  b)  the  range  of  complexity 
of  the  most  complex  tasks  for  the  occupation. 

System  dependency  is  a  problem  in  some  of  the  aircraft  since  the  engine  start 
requires  an  auxiliary  power  unit,  and  if  it  has  a  second  engine,  the  second  one  also 
may  depend  on  the  first  one’s  turning. 
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User  friendly  aircraft  systems  improve  safety  and  handling  qualities  of  the  air¬ 
craft.  User-friendly  control  systems  should  shape  handling  qualities  of  an  aircraft 
in  such  a  way  that  control  becomes  easy  and  safe.  The  autopilot  function  reduces 
the  complexity  of  interactions  between  aircraft  attitudes,  power  settings,  and  rate 
of  motion,  and  in  conclusion,  limits  the  possibility  of  loss  of  control.  The  throttle 
types  which  definitely  contribute  to  improving  flying  safety  as  well  as  being  a  user 
friendly  aircraft  control  are:  MLEC  (Multi-Lever  Engine  Control  System),  Mechanical 
SLPC  (Single-Lever  Power  Control),  Digital  fly- by-wire  SLPC  and  Combined  SLPC. 
An  SLPC  system  increases  engine  performance  and  fuel  efficiency  while  substantially 
reducing  pilot  workload  and  increasing  flight  safety  in  comparison  to  older  and  less 
user-friendly,  multi-lever  engine  control  systems.  The  benefits  of  using  SLPCs  for 
general  aviation  aircraft  are  to  improve  flight  safety  through  advanced  engine  diag¬ 
nostics,  simplify  powerplant  operations,  increase  time  between  overhauls,  and  provide 
cost-effective  technology  (extends  fuel  burn  and  reduces  overhaul  costs). 

Upgradeability  is  another  issue  for  the  aging  aircraft  fleet.  Some  levels  of  this 
are  software  upgradeability,  modular  design  improved  LRLI  (Line- Replaceable  Unit), 
modular  open  system,  and  built-in  upgradeability.  Software  upgradeability  includes 
the  systems  such  as  flight  director,  weather  (WX),  and  approach  chart  display.  Mod¬ 
ular  design  improved  LR.U  modules  enhance  serviceability  and  interchangeability.  A 
modular  open  system  uses  standard,  well-defined  interfaces  (i.e.  electrical,  mechani¬ 
cal,  software)  and  eliminates  proprietary  point  designs.  Built-in  upgradeability  is  the 
ability  to  update  entire  aircraft  quickly  and  efficiently  without  the  usual  downtime 
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and  panel  teardowns  for  traditional  avionics  enhancement.  ”  Modular  Design  means 
a  design  where  functionality  is  partitioned  into  discrete,  cohesive,  and  self-contained 
units  with  well-defined,  open  and  published  interfaces  that  permit  substitution  of  such 
units  with  similar  components  or  products  from  alternate  sources  with  minimum  im¬ 
pact  on  existing  units” .  (A  Modular  Open  Systems  Approach  (MOSA)  to  Acquisition 
document,  Open  Systems  Joint  Task  Force  (OSJTF)). 

Deactivation  capability  is  a  feature  especially  used  by  instructor  pilots  in  order 
to  disable  some  capabilities  of  the  aircraft  for  the  training  of  the  student  to  fly  without 
these  systems.  This  necessitates  that  the  student  practise  using  back-up  systems  and 
improve  his  skills  in  case  of  a  discrepancy.  Those  systems,  which  have  deactivation 
capability,  are  the  intercom/radio  system,  multi-function  display  (MFD)  modes,  the 
stall  warning  system,  and  some  of  the  navigational  aid  systems  (NAVAIDS)  . 

The  ninth  criteria,  supporting  systems,  includes  synthetic  trainer  systems,  de¬ 
briefing  systems  and  life  support  materials  (including  G-suit). 

Synthetic  training  is  recognized  as  being  important  for  aircraft  training.  A 
combination  of  synthetic  training  and  training  in  a  real  aircraft,  makes  for  a  better, 
safer  pilot.  According  to  the  studies,  pilots  who  receive  regular,  recurrent  synthetic 
training  have  far  lower  accident  rates  (and  in  the  particular  study  conducted,  not  one 
of  the  randomly  selected  pilots  had  been  involved  in  a  fatal  accident,  which  yielded 
a  probability  of  less  than  one  percent  of  occurring).  Here  are  the  synthetic  training 
systems  mentioned  in  this  study: 
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Part-Task  Trainers  (PTT):  PTTs  feature  high-fidelity  simulation  models,  pro¬ 
viding  a  realistic  environment  for  effective  training  independent  of  the  actual  training 
aircraft.  PTTs  offer  a  flexible,  economical  means  of  training  students  on  specific  air¬ 
craft  devices  and  subsystems.  Thus,  the  student  knows  how  best  to  handle  and  use 
complex  systems  such  as  avionics  or  mission  management  equipment  before  starting 
simulator  or  aircraft  missions. 

Flight  and  Navigation  Procedures  Trainer  (FNPT):  The  main  role  of  the  FNPT 
is  to  enable  the  student  pilot  to  practise  instrument  flying  and  real-world  naviga¬ 
tion.  Accordingly,  the  FNPT  is  designed  and  configured  to  integrate  all  flight  and 
navigation  systems.  The  practices  of  limited  ground  operations,  take-off,  departure, 
in-flight  manoeuvres,  radio  navigation,  GPS  operations,  approach,  landings,  and  ma¬ 
jor  emergency  procedures  can  be  executed  with  reference  to  aircraft  instruments  and 
navigation  controls.  The  FNPT  is  an  inexpensive  means  of  providing  high  perfor¬ 
mance  training  in  a  safe  environment,  completely  independent  of  weather  conditions 
and  aircraft  availability. 

Flight  Training  Device  (FTD):  FTD  enables  the  student  pilot  to  learn,  develop, 
and  practise  normal,  abnormal,  and  emergency  aircraft  specific  procedures.  In  addi¬ 
tion,  the  FTD  allows  the  student  to  gain  essential  IFR  training  and  system  manage¬ 
ment  skills  necessary  for  understanding  and  operating  modern  interactive  navigation 
and  mission  avionics  systems.  Depending  on  the  type  of  visual  system  and  database 
selected,  the  FTD  can  also  provide  training  for  VFR,  Night  Flight,  Formation  Flight, 
Low  Level  Flight  and  Mission  Rehearsal. 
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Full  Simulator:  Full  simulator  includes  features  such  as  motion  platform,  g-seat 
and  vibration  platform,  anti-g  suit,  wide  visual  system  field  of  view,  and  simula¬ 
tion  models  enhancement,  elements  which  are  considered  optional  with  the  FTD.  It 
replicates  the  aircraft’s  flight  and  ground-handling  characteristics  to  a  realistic  level, 
integrating  the  latest  in  visual  realism  and  high-fidelity  avionics  simulation  technolo¬ 
gies. 

Debriefing  systems  comprise  GPS  data  recorders,  digital  video  debriefing  sys¬ 
tems  and  ACMI  (Air  Combat  Maneuvering  Instrumentation). 

GPS  Data  Recorder:  The  GPS  data  recorder  is  a  portable  device  which  may  be 
hand  carried  or  installed  permanently  in  the  aircraft.  After  the  flight,  the  flight  track 
is  downloaded  and  can  be  used  as  material  for  the  flight  debriefing. 

Digital  Video  Debriefing  System  (DVDS):  Digital  Video  Debriefing  System  is 
an  advanced  PC-based  application  for  producing  highly  effective,  comprehensive  real¬ 
time  and  post-flight  analysis  and  debriefing  of  video  and  audio. 

ACMI:  ACMI  is  an  on-board  pod  carried  on  a  missile  launcher  which  sends 
real  time  flight  data  and  on-board  pickle  signals  to  the  ground-based  station  for  both 
online  and  after  flight  briefing. 

Life  support  materials  are  CSU-13  B/P  anti-G  suit  (standard)  and  advanced 
technology  anti-G  suit. 

Finally,  the  last  criteria  is  the  cost,  which  is  a  primary  and  important  criteria 
in  most  of  the  procurement  process  as  it  is  in  the  nature  of  the  job.  Operating  an 
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aircraft  for  training  is  always  going  to  be  expensive.  However,  for  primary  training,  a 
cheaper  aircraft  and  low-cost  operation  is  clearly  required.  The  most  important  cost 
figure  is  the  life  cycle  cost.  In  addition  to  this,  there  are  also  secondary  costs  such  as 
maintenance  and  logistic  support  costs,  and  costs  for  training  aids  such  as  simulators 
and  other  equipment,  possibly  including  airfield  facilities.  Since  cost  increases  gradu¬ 
ally  from  primary  to  advanced  training,  any  exercises  that  can  be  applied  to  an  earlier 
phase  will  be  a  cost  saving  factor  within  the  overall  training  system.  This  not  only 
applies  from  one  aircraft  type  to  another,  but  also  from  aircraft  to  simulation  devices, 
because,  giving  the  training  in  a  simulator  is  always  much  cheaper  than  operating  the 
same  training  in  an  aircraft.  If  techniques  and  systems  can  be  taught  at  the  primary 
training  aircraft  level  rather  than  in  the  advanced  training  aircraft  (or  transferred  to 
simulators),  a  large  overall  saving  could  be  realized.  Furthermore,  if  the  aircraft  flying 
hours  can  be  reduced,  the  service  life  of  the  aircraft  fleet  can  be  extended.  That  can 
be  a  huge  saving  in  the  long  term  since  the  fleet  replacement  is  not  only  very  costly 
but  is  also  politically  sensitive,  increasing  with  the  complexity  and  the  capability  of 
the  aircraft  involved.  The  aircraft  cost  has  been  measured  in  million  dollars  in  this 
study. 

The  maintenance  cost,  which  is  measured  in  hundred  dollars,  is  the  total  of  the 
labor  and  material  cost  per  flight  hour.  Military  base  or  line  maintenance  includes 
general  support  man-hours  cost  but  not  depot  or  material  costs  as  in  the  commercial 
sector. 
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Appendix  D.  Sensitivity  Analysis 

D.l  Step  9:  Sensitivity  Analysis 

Sensitivity  analysis  is  implemented  to  verify  that  the  value  models  are  built 
on  proper  assumptions.  One  of  the  biggest  assumptions  in  the  models  is  that  the 
evaluation  measures  have  been  given  the  proper  weighting  and  accurately  shows  the 
decision  makers  preferences.  Sensitivity  analysis  helps  the  decision  maker  verify  these 
weightings  by  showing  how  the  ranking  of  alternatives  may  change  based  on  variations 
in  measure  weights.  The  weight  of  a  single  value  is  varied,  while  the  weights  of  the 
remaining  values  remain  proportional.  The  sum  of  the  values  in  each  tier  will  still 
sum  to  1.  A  graph  is  generated  that  shows  how  the  alternative  ranking  will  change 
with  respect  to  variation  in  this  value.  This  is  useful  for  several  reasons.  First,  the 
decision  makers  may  have  made  errors  in  estimating  or  communicating  their  weights 
in  the  hierarchy.  Second,  external  changes  can  change  the  weights  of  the  hierarchy. 
Rather  than  having  to  perform  the  entire  analysis  again,  sensitivity  analysis  lets  the 
decision  maker  see  how  a  different  weight  would  change  the  results. 

There  are  two  basic  methods  of  examining  the  sensitivity  of  the  alternatives 
to  changes  in  the  weights  of  the  value  or  measures.  The  first  is  a  global  sensitivity 
analysis,  where  the  weight  of  the  value  or  measure  of  interest  is  varied  while  all  of  the 
other  weights  in  the  hierarchy  vary  proportionally.  The  second  is  a  local  sensitivity 
analysis,  where  the  weight  of  the  value  or  measure  of  interest  is  varied,  while  all  of  the 
weights  of  the  values  in  the  same  tier  of  the  hierarchy  vary  proportionally.  This  type 
of  sensitivity  analysis  begins  by  moving  a  selected  measures  weight  from  zero  to  one, 
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regardless  of  the  predetermined  weight.  As  the  measures  weight  changes,  the  weights 
of  all  other  evaluation  measures  are  proportionally  adjusted  to  ensure  all  weights  still 
sum  to  one.  Each  alternative  receives  more  or  less  value  depending  on  the  weight  of 
the  selected  evaluation  measure. 

Based  on  the  data  collected  for  the  alternatives  that  have  been  identified  for  the 
primary  training  aircraft,  some  measures  and  values  will  be  less  sensitive  than  others. 
The  measures  , which  are  the  same  or  similar  across  all  alternatives,  should  not  be  as 
sensitive  to  changes  in  the  weight  as  other  measures.  The  significant  findings  of  the 
sensitivity  analysis  are  explained  below. 

Figure  1  shows  the  global  sensitivity  analysis  for  the  top  tier  value  ’’Flying 
Quality”.  This  graph  shows  that  a  minor  increase  in  the  current  weight  results  in 
the  top  alternative  changes  from  T-6A  to  T-37B.  Therefore,  the  alternatives  are  not 
very  sensitive  to  the  changes  in  weight,  ft  is  only  for  the  lower  weights  that  the 
top  alternatives  change.  Right  after  0.245  range  for  the  current  weight,  the  top 
alternatives  remains  the  same.  If  ’’Flight  Quality”  becomes  less  important  to  the 
group,  alternatives  that  do  poorly  in  ’’Flight  Quality”  but  well  in  the  other  measures 
may  become  more  preferred.  Another  important  factor  to  notice  is  the  fact  that, 
the  more  the  weight  increases  the  more  the  values  of  the  alternatives  increase.  We 
should  realize  that  the  ’’Flying  Quality”  is  one  of  the  most  outweighting  values  of  all 
throughout  the  group  hierarchy.  Thus,  all  the  alternative  values  increase  more  or  less 
with  the  increase  in  this  weight. 
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Figure  1:  Global  Sensitivity  of  Flying  Quality  for  the  Group 


The  sensitivity  analysis  for  ’’Training  Quality”,  shown  in  Figure  2  ,  shows  how 
stable  are  the  alternatives  as  the  weight  is  varied.  This  graph  shows  that  for  all  the 
weights,  the  top  alternatives  remain  the  same.  The  alternatives  are  not  very  sensitive 
to  the  changes  in  weight. 

The  sensitivity  analysis  for  ’’Design”,  shown  in  Figure  3  ,  shows  how  the  worst 
alternative  at  the  beginning  gains  value  as  the  current  weight  increases.  After  the 
range  of  0.31  of  current  weight,  the  top  alternatives  are  not  sensitive;  however,  when 
the  weight  changes  from  0.126  to  0.165,  then  the  ranking  of  the  worst  alternatives 
starts  to  change. 

The  sensitivity  analysis  for  ’’Performance”,  shown  in  Figure  4  ,  shows  that  for 
the  range  immediately  surrounding  the  current  weight,  the  top  alternatives  are  not 
sensitive;  however,  if  the  weight  doubles  from  0.105  to  0.210,  then  the  ranking  of 
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Figure  2:  Global  Sensitivity  of  Training  Quality  for  the  Group 
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Figure  3:  Global  Sensitivity  of  Design  for  the  Group 

the  alternatives  starts  to  change.  After  the  range  of  0.31  of  current  weight,  the  top 
alternatives  are  not  sensitive;  however,  when  the  weight  changes  from  0.126  to  0.165, 
then  the  ranking  of  the  worst  alternatives  starts  to  change.  The  values  for  the  air- 
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crafts  converge  into  two  separate  areas.  First,  relatively  old  version  primary  training 
aircrafts  T-34C  and  T-35,  increase  dramatically  in  value  as  the  weight  increases  and 
converge  at  a  value  of  1.  The  only  double-engine  primary  trainer  T-37B  converge  at 
a  value  of  0  even  though  it  starts  with  the  most  value  at  the  beginning  of  the  current 
weight  scale.  This  graph  shows  that  the  alternatives  are  very  sensitive  to  increases  in 
weight  for  ”  Performance” . 


Figure  4:  Global  Sensitivity  of  Performance  for  the  Group 


The  sensitivity  analysis  for  ”  Safety” ,  shown  in  Figure  5  ,  shows  very  interestingly 
the  only  alternative,  T-37,  increases  and  all  other  alternatives  decreases  in  value  as  the 
weight  increases.  When  we  look  the  top-tier  value  ’’Safety”  in  Figure  A.l;  ’’Engine”, 
’’Ejection  Seat”  and  ’’Collision  Avoidance  System”  are  the  sub-tier  values.  ’’Engine” 
,  with  the  highest  local  weight  of  0.625,  contributes  dramatically  to  T-37B  increase 
in  value  since  it’s  the  only  double-engine  primary  training  aircraft. 
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Global  Sensitivity  of  SAFETY 


Figure  5:  Global  Sensitivity  of  Safety  for  the  Group 


The  sensitivity  analysis  for  ”  Maintainability” ,  shown  in  Figure  6,  shows  that  all 
the  alternatives  decrease  in  value  as  the  weight  increases.  At  the  current  weight,  the 
best  alternative  changes  from  T-37B  to  T-6A  and  remains  the  same  to  the  highest 
weight. 

The  sensitivity  analysis  for  ’’Supporting  Systems”,  shown  in  Figure  7,  shows 
the  sensitivity  for  the  ’’Synthetic  Trainer  Systems”  and  ’’Debriefing  Systems”.  For 
the  range  immediately  surrounding  the  current  weight,  the  top  alternatives  are  not 
sensitive;  however,  if  the  weight  doubles  from  0.053  to  0.1,  then  the  ranking  of  the 
alternatives  starts  to  change.  The  values  for  the  alternative  primary  training  aircrafts 
converge  into  three  separate  areas.  First,  the  newest  generation  trainers  T-6A  and 
KT-1C,  those  with  the  full  simulators  and  digital  video  debriefing  systems,  increase 
dramatically  in  value  as  the  weight  increases  and  converge  at  a  value  of  0.96  .  The 
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Global  Sensitivity  of  MAINTAINABILITY 


Figure  6:  Global  Sensitivity  of  Maintainability  for  the  Group 


aircrafts  with  the  flight  &  navigation  procedures  trainers  converge  at  a  value  of  0.28 
and  the  aircrafts  without  a  synthetic  trainer  and  a  debriefing  system  converge  at  a 
value  of  0. 

The  sensitivity  analysis  for  ’’Cost”,  shown  in  Figure  8,  shows  how  the  groups 
weights  the  ’’Cost”  to  the  minimum  extent  in  comparison  to  the  other  values.  If  the 
current  weight  increases  from  0.011  to  0.13,  the  alternatives  change  dramatically  in 
value.  The  newest  generation  aircrafts  T-6A  and  KT-1C  converges  at  a  value  of  0  as 
the  weight  increases  to  1  since  they  have  the  highest  unit  cost.  Even  though  T-35 
has  the  lowest  unit  cost,  T-37B  gets  the  highest  value  due  to  lower  maintenance  cost 
than  T-35  when  the  weight  is  increased  to  0.21  and  then. 

The  sensitivity  analysis  for  ’’Technology”,  shown  in  Figure  9  ,  shows  the  sen¬ 
sitivity  for  the  consistency  of  the  systems  on-board  to  the  current  technology.  For 
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Global  Sensitivity  of  SUPPORTING  SYSTEMS 


Figure  7:  Global  Sensitivity  of  Supporting  Systems  for  the 

Group 


Figure  8:  Global  Sensitivity  of  Cost  for  the  Group 


the  range  immediately  surrounding  the  current  weight,  the  top  alternatives  are  not 


sensitive;  however,  if  the  weight  changes  from  0.03  to  0.1,  then  the  ranking  of  the 


alternatives  starts  to  change.  The  values  for  the  aircrafts  converge  into  three  separate 
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areas.  First,  T-6A,  KT-1C  and  EMB.312,  with  the  most  current  technology,  increase 
dramatically  in  value  as  the  weight  increases  and  converge  at  a  value  of  1.  T-37B 
relatively  keeps  its  value  with  the  changing  weight  and  converges  at  a  value  of  0.63 
and  T-34C  and  T-35,  with  the  earliest  technology  on-board,  converge  at  a  value  of 
0.48  . 


Figure  9:  Global  Sensitivity  of  Technology  for  the  Group 
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